philipp/Abschluss-Projekt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: philipp:55b1b2b5feea5803b41b34416fd78741bf808567
Branches Tags
philipp:main
...
compare: philipp:main
Branches Tags
philipp:main

7 Commits
55b1b2b5fe ... main

Author SHA1 Message Date
Crussade
14ad709b53 gitgnore 2025-12-08 12:51:50 +01:00
Crussade
ccfb40a2b3 cleanup add training bell 2025-12-08 12:26:34 +01:00
Philipp
036f3b178a que and training status 2025-12-02 12:50:22 +01:00
Philipp
5bfe82fc26 addet training info 2025-12-02 10:51:20 +01:00
Philipp
de5a8d2028 add multi trainings +fix path for coco retraining 2025-12-02 10:16:18 +01:00
Philipp
0e31237b79 training fix.add global settings 2025-12-02 09:55:50 +01:00
Crussade
c3c7e042bb training fix. add global settings 2025-12-02 09:31:52 +01:00
3079 changed files with 679039 additions and 7307 deletions
Expand all files Collapse all files

2
.gitignore vendored Normal file → Executable file
View File

@@ -3,3 +3,5 @@ backend/uploads/*.pth
*.pth *.pth
backend/node_modules/ backend/node_modules/
backend/.venv/ backend/.venv/
key
key.pub

217
TRANSFER_LEARNING_FEATURE.md
View File

@@ -1,217 +0,0 @@
# Transfer Learning Base Configuration Feature
## Overview
This feature implements automatic loading of base configurations when "Train on COCO" transfer learning is selected. Base parameters are loaded from `backend/data/` based on the selected YOLOX model, and these protected fields are displayed as greyed out and non-editable in the frontend.
## Components Modified/Created
### Backend
#### 1. Base Configuration Files (`backend/data/`)
- **`yolox_s.py`** - Base config for YOLOX-Small (depth=0.33, width=0.50)
- **`yolox_m.py`** - Base config for YOLOX-Medium (depth=0.67, width=0.75)
- **`yolox_l.py`** - Base config for YOLOX-Large (depth=1.0, width=1.0)
- **`yolox_x.py`** - Base config for YOLOX-XLarge (depth=1.33, width=1.25)
Each file contains a `BaseExp` class with protected parameters:
- Model architecture (depth, width, activation)
- Training hyperparameters (max_epoch, warmup_epochs, scheduler, etc.)
- Optimizer settings (momentum, weight_decay)
- Augmentation probabilities (mosaic_prob, mixup_prob, etc.)
- Input/output sizes
#### 2. Services (`backend/services/generate_yolox_exp.py`)
**New functions:**
- `load_base_config(selected_model)` - Dynamically loads base config using importlib
- Modified `generate_yolox_inference_exp()` to support `use_base_config` parameter
- Base config merging logic: base → user overrides → defaults
**Behavior:**
- `transfer_learning='coco'` → loads base config + applies user overrides
- `transfer_learning='sketch'` → uses only user-defined values
- Protected parameters from base config are preserved unless explicitly overridden
#### 3. API Routes (`backend/routes/api.py`)
**New endpoint:**
```python
@api_bp.route('/base-config/<model_name>', methods=['GET'])
def get_base_config(model_name):
```
Returns the base configuration JSON for a specific YOLOX model.
### Frontend
#### 1. HTML (`edit-training.html`)
**Added:**
- Info banner to indicate when base config is active
- CSS styles for disabled input fields (grey background, not-allowed cursor)
- Visual feedback showing which model's base config is loaded
**Banner HTML:**
```html
<div id="base-config-info" style="display:none; ...">
🔒 Base Configuration Active
Protected parameters are loaded from [model] base config
</div>
```
**CSS for disabled fields:**
```css
.setting-row input[type="number"]:disabled,
.setting-row input[type="text"]:disabled,
.setting-row input[type="checkbox"]:disabled {
background: #d3d3d3 !important;
color: #666 !important;
cursor: not-allowed !important;
border: 1px solid #999 !important;
}
```
#### 2. JavaScript (`js/start-training.js`)
**New functionality:**
1. **Base Config Loading:**
```javascript
function loadBaseConfig(modelName)
```
Fetches base config from `/api/base-config/<model>`
2. **Apply Base Config:**
```javascript
function applyBaseConfig(config, isCocoMode)
```
- Applies config values to form fields
- Disables and greys out protected fields
- Shows/hides info banner
- Adds tooltips to disabled fields
3. **Update Transfer Learning Mode:**
```javascript
function updateTransferLearningMode()
```
- Monitors changes to "Transfer Learning" dropdown
- Monitors changes to "Select Model" dropdown
- Loads appropriate base config when COCO mode is selected
- Clears base config when sketch mode is selected
4. **Form Submission Enhancement:**
- Temporarily enables disabled fields before submission
- Ensures protected parameters are included in form data
- Re-disables fields after collection
**Protected Fields List:**
```javascript
const protectedFields = [
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
];
```
## User Flow
### 1. Normal Custom Training (Train from sketch)
- User selects model: e.g., "YOLOX-s"
- User selects "Train from sketch"
- All fields are editable (white background)
- User can customize all parameters
- Submission uses user-defined values only
### 2. COCO Transfer Learning (Train on COCO)
- User selects model: e.g., "YOLOX-s"
- User selects "Train on coco"
- **Automatic actions:**
1. Frontend calls `/api/base-config/YOLOX-s`
2. Base config is loaded and applied
3. Protected fields become greyed out and disabled
4. Green info banner appears: "🔒 Base Configuration Active"
5. Tooltip on hover: "Protected by base config for YOLOX-s. Switch to 'Train from sketch' to customize."
- User can still edit non-protected fields
- On submit: both base config values AND user overrides are sent to backend
- Backend generates exp.py with merged settings
### 3. Switching Models
- User changes from "YOLOX-s" to "YOLOX-l" (while in COCO mode)
- Frontend automatically:
1. Fetches new base config for YOLOX-l
2. Updates field values (depth=1.0, width=1.0, etc.)
3. Updates banner text to show "YOLOX-l"
- Protected parameters update to match new model's architecture
## Testing
### Manual Test Steps:
1. **Test Base Config Loading:**
```bash
cd backend/data
python test_base_configs.py
```
Should display all parameters for yolox-s, yolox-m, yolox-l, yolox-x
2. **Test API Endpoint:**
```bash
# Start Flask server
cd backend
python app.py
# In another terminal:
curl http://localhost:3000/api/base-config/YOLOX-s
```
Should return JSON with depth, width, activation, etc.
3. **Test Frontend:**
- Open `edit-training.html?id=<project_id>` in browser
- Select "YOLOX-s" model
- Select "Train on coco" → fields should grey out
- Select "Train from sketch" → fields should become editable
- Switch to "YOLOX-l" (in COCO mode) → values should update
- Open browser console and check for: `Applied base config. Protected fields: depth, width, ...`
4. **Test Form Submission:**
- With COCO mode active (fields greyed out)
- Click "Save Parameters"
- Check browser Network tab → POST to `/api/yolox-settings`
- Verify payload includes protected parameters (depth, width, etc.)
- Check Flask logs for successful save
### Expected Behaviors:
✅ **COCO mode + YOLOX-s:**
- depth: 0.33 (greyed out)
- width: 0.50 (greyed out)
- activation: silu (greyed out)
- Info banner visible
✅ **COCO mode + YOLOX-l:**
- depth: 1.0 (greyed out)
- width: 1.0 (greyed out)
- activation: silu (greyed out)
✅ **Sketch mode:**
- All fields white/editable
- No info banner
- User can set any values
## Documentation
- **`backend/data/README.md`** - Complete guide on base config system
- **`backend/data/test_base_configs.py`** - Test script for base configs
## Benefits
1. **Proven defaults:** Users start with battle-tested COCO pretraining settings
2. **Prevents mistakes:** Can't accidentally break model architecture by changing depth/width
3. **Easy customization:** Can still override specific parameters if needed
4. **Visual feedback:** Clear indication of which fields are protected
5. **Model-specific:** Each model (s/m/l/x) has appropriate architecture defaults
6. **Flexible:** Can easily add new models by creating new base config files
## Future Enhancements
- Add "Override" button next to protected fields to unlock individual parameters
- Show diff comparison between base config and user overrides
- Add validation warnings if user tries values far from base config ranges
- Export final merged config as preview before training

538
add-project.html Normal file → Executable file
View File

@@ -1,175 +1,365 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body> <body>
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon> style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" class="button-red">Add Training Project</button> <!-- Training Notification Bell -->
<button id="Add Dataset" class="button">Add Dataset</button> <button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
<button id="Import Dataset" class="button">Refresh Label-Studio</button> style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
<button id="seed-db-btn" class="button">Seed Database</button> 🔔
</div> <span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
</div> color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
text-align: center; font-weight: bold;">0</span>
<div class="popup"> </button>
<div class="upload-button" onclick="uploadButtonHandler()"> <button id="Add Training Project" class="button-red">Add Training Project</button>
<span class="upload-button-text">Upload</span> <button id="Add Dataset" class="button">Add Dataset</button>
</div> <button id="Import Dataset" class="button">Refresh Label-Studio</button>
<div class="image"></div> <button id="seed-db-btn" class="button">Seed Database</button>
<button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
<div class="div"> </div>
</div>
<div class="add-category">
<div class="popup">
<input class="div-wrapper" placeholder="Class name"></input> <div class="upload-button" onclick="uploadButtonHandler()">
<button class="upload-button-text-wrapper"> <span class="upload-button-text">Upload</span>
<span class="button-text-upload">Add Class</span> </div>
</button> <div class="image"></div>
</div>
<input class="project-name" placeholder="Project Name" id="project_name_input"></input> <div class="div">
<textarea class="add-description" placeholder="Description" id="project_description_input"></textarea>
<div class="add-class-wrapper"> <div class="add-category">
<script type="module">
import { addClass } from './js/add-class.js'; <input class="div-wrapper" placeholder="Class name"></input>
addClass(); <button class="upload-button-text-wrapper">
</script> <span class="button-text-upload">Add Class</span>
</div> </button>
</div> </div>
<button class="confirm-button-datasetcreation"> <input class="project-name" placeholder="Project Name" id="project_name_input"></input>
<span class="button-text-upload">Confirm</span> <textarea class="add-description" placeholder="Description" id="project_description_input"></textarea>
</button> <div class="add-class-wrapper">
</div> <script type="module">
</div> import { addClass } from './js/add-class.js';
</div> addClass();
</script>
</div> </div>
<script type="text/javascript" src="./js/add-image.js"></script> </div>
<script type="module"> <button class="confirm-button-datasetcreation">
import { addClass } from './js/add-class.js'; <span class="button-text-upload">Confirm</span>
// Grab the inputs and the button </button>
const projectNameInput = document.querySelector('.project-name'); </div>
const descriptionInput = document.querySelector('.add-description'); </div>
const confirmButton = document.querySelector('.confirm-button-datasetcreation'); </div>
const classWrapper = document.querySelector('.add-class-wrapper');
</div>
<script type="text/javascript" src="./js/add-image.js"></script>
const addClassButton = document.querySelector('.upload-button-text-wrapper'); <script type="module">
const addProjectButton = document.querySelector('.confirm-button-datasetcreation') import { addClass } from './js/add-class.js';
// Grab the inputs and the button
addClassButton.addEventListener('click', () => { const projectNameInput = document.querySelector('.project-name');
const event = new Event('classListUpdated'); const descriptionInput = document.querySelector('.add-description');
document.dispatchEvent(event); const confirmButton = document.querySelector('.confirm-button-datasetcreation');
}); const classWrapper = document.querySelector('.add-class-wrapper');
const addClassButton = document.querySelector('.upload-button-text-wrapper');
const addProjectButton = document.querySelector('.confirm-button-datasetcreation')
// Function to update button state addClassButton.addEventListener('click', () => {
function updateButtonState() { const event = new Event('classListUpdated');
const projectName = projectNameInput.value.trim(); document.dispatchEvent(event);
const description = descriptionInput.value.trim(); });
const existingClasses = classWrapper.querySelectorAll('.overlap-group');
// Disable button if either field is empty
if (projectName === '' || description === '' || existingClasses.length === 0) {
confirmButton.disabled = true;
confirmButton.style.cursor = 'not-allowed'; // Function to update button state
} else { function updateButtonState() {
confirmButton.disabled = false; const projectName = projectNameInput.value.trim();
confirmButton.style.cursor = 'pointer'; const description = descriptionInput.value.trim();
} const existingClasses = classWrapper.querySelectorAll('.overlap-group');
}
// Disable button if either field is empty
// Initial check on page load if (projectName === '' || description === '' || existingClasses.length === 0) {
updateButtonState(); confirmButton.disabled = true;
confirmButton.style.cursor = 'not-allowed';
} else {
projectNameInput.addEventListener('input', updateButtonState); confirmButton.disabled = false;
descriptionInput.addEventListener('input', updateButtonState); confirmButton.style.cursor = 'pointer';
document.addEventListener('classListUpdated', updateButtonState); }
</script> }
<script> // Initial check on page load
document.getElementById('seed-db-btn').addEventListener('click', function () { updateButtonState();
fetch('/api/seed')
});
</script> projectNameInput.addEventListener('input', updateButtonState);
<script> descriptionInput.addEventListener('input', updateButtonState);
document.getElementById('seed-db-btn').addEventListener('click', function () { document.addEventListener('classListUpdated', updateButtonState);
const elLoader = document.getElementById("loader") </script>
elLoader.style.display = "inherit"
<script>
fetch('/api/seed') document.getElementById('seed-db-btn').addEventListener('click', function () {
.finally(() => { fetch('/api/seed')
// Instead of hiding loader immediately, poll /api/update-status until done });
function pollStatus() { </script>
fetch('/api/update-status') <script>
.then(res => res.json()) document.getElementById('seed-db-btn').addEventListener('click', function () {
.then(status => { const elLoader = document.getElementById("loader")
if (status && status.running) { elLoader.style.display = "inherit"
// Still running, poll again after short delay
setTimeout(pollStatus, 5000); fetch('/api/seed')
} else { .finally(() => {
elLoader.style.display = "none"; // Instead of hiding loader immediately, poll /api/update-status until done
} function pollStatus() {
}) fetch('/api/update-status')
.catch(() => { .then(res => res.json())
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} // Still running, poll again after short delay
pollStatus(); setTimeout(pollStatus, 5000);
}) } else {
}); elLoader.style.display = "none";
}
// Show loader if backend is still processing on page load })
.catch(() => {
function pollStatus() { elLoader.style.display = "none";
const elLoader = document.getElementById("loader"); });
fetch('/api/update-status') }
.then(res => res.json()) pollStatus();
})
.then(status => { });
if (status && status.running) {
elLoader.style.display = "inherit"; // Show loader if backend is still processing on page load
setTimeout(pollStatus, 5000);
} else { function pollStatus() {
elLoader.style.display = "none"; const elLoader = document.getElementById("loader");
} fetch('/api/update-status')
}) .then(res => res.json())
.catch(() => {
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} elLoader.style.display = "inherit";
setTimeout(pollStatus, 5000);
</script> } else {
elLoader.style.display = "none";
}
</body> })
.catch(() => {
elLoader.style.display = "none";
});
}
</script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
</div>
</div>
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
</div>
</div>
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html> </html>

80
backend/.gitignore vendored Normal file → Executable file
View File

@@ -1,40 +1,40 @@
# Python # Python
__pycache__/ __pycache__/
*.py[cod] *.py[cod]
*$py.class *$py.class
*.so *.so
.Python .Python
venv/ venv/
env/ env/
ENV/ ENV/
*.egg-info/ *.egg-info/
dist/ dist/
build/ build/
# Flask # Flask
instance/ instance/
.webassets-cache .webassets-cache
# Database # Database
*.db *.db
*.sqlite *.sqlite
# Environment # Environment
.env .env
.flaskenv .flaskenv
# IDE # IDE
.vscode/ .vscode/
.idea/ .idea/
*.swp *.swp
*.swo *.swo
# OS # OS
.DS_Store .DS_Store
Thumbs.db Thumbs.db
# Logs # Logs
*.log *.log
# Uploads # Uploads
uploads/*.pth uploads/*.pth

28
backend/0815/27/exp_infer.py
View File

@@ -1,28 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_27_train.json"
self.val_ann = "coco_project_27_valid.json"
self.test_ann = "coco_project_27_test.json"
self.num_classes = 80
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.random_size = (10, 20)
self.test_size = (640.0, 640.0)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

148
backend/CONVERSION_SUMMARY.md
View File

@@ -1,148 +0,0 @@
# Backend Conversion Summary
## ✅ Conversion Complete
Your Node.js backend has been successfully converted to Python using Flask and SQLAlchemy.
## 📁 New Python Files Created
### Core Application
- **app.py** - Main Flask application (replaces server.js)
- **start.py** - Startup script for easy launching
- **requirements.txt** - Python dependencies (replaces package.json)
### Database Layer
- **database/database.py** - SQLAlchemy database configuration (replaces database.js)
### Models (Sequelize → SQLAlchemy)
- **models/TrainingProject.py**
- **models/TrainingProjectDetails.py**
- **models/training.py**
- **models/LabelStudioProject.py**
- **models/Images.py**
- **models/Annotation.py**
- **models/__init__.py**
### API Routes
- **routes/api.py** - All API endpoints converted to Flask blueprints (replaces api.js)
- **routes/__init__.py**
### Services
- **services/fetch_labelstudio.py** - Label Studio API integration
- **services/seed_label_studio.py** - Database seeding logic
- **services/generate_json_yolox.py** - COCO JSON generation
- **services/generate_yolox_exp.py** - YOLOX experiment file generation
- **services/push_yolox_exp.py** - Save training settings to DB
- **services/__init__.py**
### Documentation
- **README.md** - Comprehensive documentation
- **QUICKSTART.md** - Quick setup guide
- **.gitignore** - Python-specific ignore patterns
## 🔄 Key Changes
### Technology Stack
| Component | Node.js | Python |
|-----------|---------|--------|
| Framework | Express.js | Flask |
| ORM | Sequelize | SQLAlchemy |
| HTTP Client | node-fetch | requests |
| Package Manager | npm | pip |
| Runtime | Node.js | Python 3.8+ |
### API Compatibility
✅ All endpoints preserved with same URLs
✅ Request/response formats maintained
✅ Same database schema
✅ Same business logic
### Converted Features
- ✅ Training project management
- ✅ Label Studio integration
- ✅ YOLOX configuration and training
- ✅ File upload handling
- ✅ Image and annotation management
- ✅ COCO JSON generation
- ✅ Training logs
## 🚀 Getting Started
1. **Install dependencies:**
```bash
cd backend
python -m venv venv
.\venv\Scripts\Activate.ps1 # Windows
pip install -r requirements.txt
```
2. **Run the server:**
```bash
python start.py
```
3. **Server runs at:** `http://0.0.0.0:3000`
## 📦 Dependencies Installed
- Flask 3.0.0 - Web framework
- Flask-CORS 4.0.0 - Cross-origin resource sharing
- Flask-SQLAlchemy 3.1.1 - ORM integration
- SQLAlchemy 2.0.23 - Database ORM
- PyMySQL 1.1.0 - MySQL driver
- requests 2.31.0 - HTTP client
- Pillow 10.1.0 - Image processing
## ⚠️ Important Notes
1. **Virtual Environment**: Always activate the virtual environment before running
2. **Database**: MySQL must be running with the `myapp` database created
3. **Credentials**: Update database credentials in `app.py` if needed
4. **Python Version**: Requires Python 3.8 or higher
## 🧪 Testing
Test the conversion:
```bash
# Get all training projects
curl http://localhost:3000/api/training-projects
# Get Label Studio projects
curl http://localhost:3000/api/label-studio-projects
```
## 📝 Original Files
Your original Node.js files remain untouched:
- server.js
- package.json
- routes/api.js
- models/*.js (JavaScript)
- services/*.js (JavaScript)
You can keep them as backup or remove them once you verify the Python version works correctly.
## 🔍 What to Verify
1. ✅ Database connection works
2. ✅ All API endpoints respond correctly
3. ✅ File uploads work
4. ✅ Label Studio integration works
5. ✅ YOLOX training can be triggered
6. ✅ COCO JSON generation works
## 🐛 Troubleshooting
See **QUICKSTART.md** for common issues and solutions.
## 📚 Further Documentation
- **README.md** - Complete project documentation
- **QUICKSTART.md** - Setup guide
- **API Documentation** - All endpoints documented in README.md
---
**Conversion completed successfully!** 🎉
Your backend is now running on Python with Flask and SQLAlchemy.

113
backend/QUICKSTART.md
View File

@@ -1,113 +0,0 @@
# Quick Start Guide - Python Backend
## Step-by-Step Setup
### 1. Install Python
Make sure you have Python 3.8 or higher installed:
```bash
python --version
```
### 2. Create Virtual Environment
```bash
cd backend
python -m venv venv
```
### 3. Activate Virtual Environment
**Windows:**
```powershell
.\venv\Scripts\Activate.ps1
```
**Linux/Mac:**
```bash
source venv/bin/activate
```
### 4. Install Dependencies
```bash
pip install -r requirements.txt
```
### 5. Verify Database Connection
Make sure MySQL is running and the database `myapp` exists:
```sql
CREATE DATABASE IF NOT EXISTS myapp;
```
### 6. Run the Server
```bash
python start.py
```
Or:
```bash
python app.py
```
The server should now be running at `http://0.0.0.0:3000`
## Testing the API
Test if the server is working:
```bash
curl http://localhost:3000/api/training-projects
```
## Common Issues
### ModuleNotFoundError
If you get import errors, make sure you've activated the virtual environment and installed all dependencies.
### Database Connection Error
Check that:
- MySQL is running
- Database credentials in `app.py` are correct
- Database `myapp` exists
### Port Already in Use
If port 3000 is already in use, modify the port in `app.py`:
```python
app.run(host='0.0.0.0', port=3001, debug=True)
```
## What Changed from Node.js
1. **Server Framework**: Express.js → Flask
2. **ORM**: Sequelize → SQLAlchemy
3. **HTTP Client**: node-fetch → requests
4. **Package Manager**: npm → pip
5. **Dependencies**: package.json → requirements.txt
6. **Startup**: `node server.js` → `python app.py`
## Next Steps
1. Test all API endpoints
2. Update frontend to point to the new Python backend (if needed)
3. Migrate any remaining Node.js-specific logic
4. Test file uploads and downloads
5. Test YOLOX training functionality
## File Structure Comparison
**Before (Node.js):**
```
backend/
├── server.js
├── package.json
├── routes/api.js
├── models/*.js
└── services/*.js
```
**After (Python):**
```
backend/
├── app.py
├── requirements.txt
├── routes/api.py
├── models/*.py
└── services/*.py
```

214
backend/README.md Normal file → Executable file
View File

@@ -1,107 +1,107 @@
# Python Backend for COCO Tool # Python Backend for COCO Tool
This is the converted Python backend using Flask and SQLAlchemy. This is the converted Python backend using Flask and SQLAlchemy.
## Setup ## Setup
1. Create a virtual environment (recommended): 1. Create a virtual environment (recommended):
```bash ```bash
python -m venv venv python -m venv venv
``` ```
2. Activate the virtual environment: 2. Activate the virtual environment:
- Windows: `venv\Scripts\activate` - Windows: `venv\Scripts\activate`
- Linux/Mac: `source venv/bin/activate` - Linux/Mac: `source venv/bin/activate`
3. Install dependencies: 3. Install dependencies:
```bash ```bash
pip install -r requirements.txt pip install -r requirements.txt
``` ```
## Running the Server ## Running the Server
### Option 1: Using start.py ### Option 1: Using start.py
```bash ```bash
python start.py python start.py
``` ```
### Option 2: Using Flask directly ### Option 2: Using Flask directly
```bash ```bash
python app.py python app.py
``` ```
### Option 3: Using Flask CLI ### Option 3: Using Flask CLI
```bash ```bash
flask --app app run --host=0.0.0.0 --port=3000 flask --app app run --host=0.0.0.0 --port=3000
``` ```
The server will start on `http://0.0.0.0:3000` The server will start on `http://0.0.0.0:3000`
## Database Configuration ## Database Configuration
The database configuration is in `database/database.py`. Default settings: The database configuration is in `database/database.py`. Default settings:
- Host: localhost - Host: localhost
- Database: myapp - Database: myapp
- User: root - User: root
- Password: root - Password: root
Modify `app.py` to change these settings. Modify `app.py` to change these settings.
## Project Structure ## Project Structure
``` ```
backend/ backend/
├── app.py # Main Flask application ├── app.py # Main Flask application
├── start.py # Startup script ├── start.py # Startup script
├── requirements.txt # Python dependencies ├── requirements.txt # Python dependencies
├── database/ ├── database/
│ └── database.py # Database configuration │ └── database.py # Database configuration
├── models/ # SQLAlchemy models ├── models/ # SQLAlchemy models
│ ├── __init__.py │ ├── __init__.py
│ ├── Annotation.py │ ├── Annotation.py
│ ├── Images.py │ ├── Images.py
│ ├── LabelStudioProject.py │ ├── LabelStudioProject.py
│ ├── training.py │ ├── training.py
│ ├── TrainingProject.py │ ├── TrainingProject.py
│ └── TrainingProjectDetails.py │ └── TrainingProjectDetails.py
├── routes/ ├── routes/
│ └── api.py # API endpoints │ └── api.py # API endpoints
└── services/ # Business logic └── services/ # Business logic
├── fetch_labelstudio.py ├── fetch_labelstudio.py
├── generate_json_yolox.py ├── generate_json_yolox.py
├── generate_yolox_exp.py ├── generate_yolox_exp.py
├── push_yolox_exp.py ├── push_yolox_exp.py
└── seed_label_studio.py └── seed_label_studio.py
``` ```
## API Endpoints ## API Endpoints
All endpoints are prefixed with `/api`: All endpoints are prefixed with `/api`:
- `GET /api/seed` - Seed database from Label Studio - `GET /api/seed` - Seed database from Label Studio
- `POST /api/generate-yolox-json` - Generate YOLOX training files - `POST /api/generate-yolox-json` - Generate YOLOX training files
- `POST /api/start-yolox-training` - Start YOLOX training - `POST /api/start-yolox-training` - Start YOLOX training
- `GET /api/training-log` - Get training logs - `GET /api/training-log` - Get training logs
- `GET/POST /api/training-projects` - Manage training projects - `GET/POST /api/training-projects` - Manage training projects
- `GET /api/label-studio-projects` - Get Label Studio projects - `GET /api/label-studio-projects` - Get Label Studio projects
- `GET/POST/PUT /api/training-project-details` - Manage project details - `GET/POST/PUT /api/training-project-details` - Manage project details
- `POST /api/yolox-settings` - Save YOLOX settings - `POST /api/yolox-settings` - Save YOLOX settings
- `GET/DELETE /api/trainings` - Manage trainings - `GET/DELETE /api/trainings` - Manage trainings
- `DELETE /api/training-projects/:id` - Delete training project - `DELETE /api/training-projects/:id` - Delete training project
## Migration Notes ## Migration Notes
This is a direct conversion from Node.js/Express to Python/Flask: This is a direct conversion from Node.js/Express to Python/Flask:
- Express → Flask - Express → Flask
- Sequelize ORM → SQLAlchemy ORM - Sequelize ORM → SQLAlchemy ORM
- node-fetch → requests library - node-fetch → requests library
- Async routes maintained where needed - Async routes maintained where needed
- All file paths and logic preserved from original - All file paths and logic preserved from original
## Differences from Node.js Version ## Differences from Node.js Version
1. Python uses async/await differently - some routes may need adjustments 1. Python uses async/await differently - some routes may need adjustments
2. File handling uses Python's built-in open() instead of fs module 2. File handling uses Python's built-in open() instead of fs module
3. Subprocess calls use Python's subprocess module 3. Subprocess calls use Python's subprocess module
4. JSON handling uses Python's json module 4. JSON handling uses Python's json module
5. Path operations use os.path instead of Node's path module 5. Path operations use os.path instead of Node's path module

91
backend/app.py Normal file → Executable file
View File

@@ -1,43 +1,48 @@
from flask import Flask, send_from_directory from flask import Flask, send_from_directory
from flask_cors import CORS from flask_cors import CORS
import os import os
from database.database import db, init_db from database.database import db, init_db
app = Flask(__name__, static_folder='..', static_url_path='') app = Flask(__name__, static_folder='..', static_url_path='')
CORS(app) CORS(app)
# Configure database # Configure database
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp' app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:root@localhost/myapp2'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
# Initialize database # Initialize database
db.init_app(app) db.init_app(app)
# Import and register blueprints # Import and register blueprints
from routes.api import api_bp from routes.api import api_bp
app.register_blueprint(api_bp, url_prefix='/api') app.register_blueprint(api_bp, url_prefix='/api')
# Serve static files (HTML, CSS, JS) # Serve static files (HTML, CSS, JS)
@app.route('/') @app.route('/')
def index(): def index():
return send_from_directory('..', 'index.html') return send_from_directory('..', 'index.html')
@app.route('/<path:path>') @app.route('/<path:path>')
def serve_static(path): def serve_static(path):
return send_from_directory('..', path) return send_from_directory('..', path)
# Initialize DB and start server # Initialize DB and start server
if __name__ == '__main__': if __name__ == '__main__':
with app.app_context(): with app.app_context():
try: try:
# Test database connection # Test database connection
db.engine.connect() db.engine.connect()
print('DB connection established.') print('DB connection established.')
# Create tables if they don't exist # Create tables if they don't exist
db.create_all() db.create_all()
# Start server # Initialize default settings
app.run(host='0.0.0.0', port=3000, debug=True) from services.settings_service import initialize_default_settings
except Exception as err: initialize_default_settings()
print(f'Failed to start: {err}') print('Settings initialized.')
# Start server
app.run(host='0.0.0.0', port=4000, debug=True)
except Exception as err:
print(f'Failed to start: {err}')

25
backend/asdf/5/exp_infer.py
View File

@@ -1,25 +0,0 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_5_train.json"
self.val_ann = "coco_project_5_valid.json"
self.test_ann = "coco_project_5_test.json"
self.num_classes = 4
self.depth = 1.0
self.width = 1.0
self.input_size = (640, 640)
self.mosaic_scale = (0.1, 2)
self.random_size = (10, 20)
self.test_size = (640, 640)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
self.enable_mixup = False

28
backend/check_db.py Normal file → Executable file
View File

@@ -1,14 +1,14 @@
import pymysql import pymysql
conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp') conn = pymysql.connect(host='localhost', user='root', password='root', database='myapp2')
cursor = conn.cursor() cursor = conn.cursor()
cursor.execute('DESCRIBE image') cursor.execute('DESCRIBE image')
rows = cursor.fetchall() rows = cursor.fetchall()
print("Current 'image' table structure:") print("Current 'image' table structure:")
print("-" * 60) print("-" * 60)
for row in rows: for row in rows:
print(f"Field: {row[0]:<15} Type: {row[1]:<15} Null: {row[2]}") print(f"Field: {row[0]:<15} Type: {row[1]:<15} Null: {row[2]}")
print("-" * 60) print("-" * 60)
conn.close() conn.close()

280
backend/data/README.md Normal file → Executable file
View File

@@ -1,140 +1,140 @@
# YOLOX Base Configuration System # YOLOX Base Configuration System
## Overview ## Overview
This directory contains base experiment configurations for YOLOX models. These configurations define "protected" parameters that are preserved during transfer learning from COCO-pretrained models. This directory contains base experiment configurations for YOLOX models. These configurations define "protected" parameters that are preserved during transfer learning from COCO-pretrained models.
## How It Works ## How It Works
### Transfer Learning Flow ### Transfer Learning Flow
1. **COCO Transfer Learning** (`transfer_learning = 'coco'`): 1. **COCO Transfer Learning** (`transfer_learning = 'coco'`):
- Loads base configuration from `data/yolox_*.py` based on `selected_model` - Loads base configuration from `data/yolox_*.py` based on `selected_model`
- Base parameters are **protected** and used as defaults - Base parameters are **protected** and used as defaults
- User settings from the form only override what's explicitly set - User settings from the form only override what's explicitly set
- Result: Best of both worlds - proven COCO settings + your customizations - Result: Best of both worlds - proven COCO settings + your customizations
2. **Sketch/Custom Training** (`transfer_learning = 'sketch'`): 2. **Sketch/Custom Training** (`transfer_learning = 'sketch'`):
- No base configuration loaded - No base configuration loaded
- Uses only user-defined parameters from the training form - Uses only user-defined parameters from the training form
- Full control over all settings - Full control over all settings
### Base Configuration Files ### Base Configuration Files
- `yolox_s.py` - YOLOX-Small (depth=0.33, width=0.50) - `yolox_s.py` - YOLOX-Small (depth=0.33, width=0.50)
- `yolox_m.py` - YOLOX-Medium (depth=0.67, width=0.75) - `yolox_m.py` - YOLOX-Medium (depth=0.67, width=0.75)
- `yolox_l.py` - YOLOX-Large (depth=1.0, width=1.0) - `yolox_l.py` - YOLOX-Large (depth=1.0, width=1.0)
- `yolox_x.py` - YOLOX-XLarge (depth=1.33, width=1.25) - `yolox_x.py` - YOLOX-XLarge (depth=1.33, width=1.25)
### Protected Parameters ### Protected Parameters
These parameters are defined in base configs and **preserved** unless explicitly overridden: These parameters are defined in base configs and **preserved** unless explicitly overridden:
**Model Architecture:** **Model Architecture:**
- `depth` - Model depth multiplier - `depth` - Model depth multiplier
- `width` - Model width multiplier - `width` - Model width multiplier
- `activation` - Activation function (silu) - `activation` - Activation function (silu)
**Training Hyperparameters:** **Training Hyperparameters:**
- `basic_lr_per_img` - Learning rate per image - `basic_lr_per_img` - Learning rate per image
- `scheduler` - LR scheduler (yoloxwarmcos) - `scheduler` - LR scheduler (yoloxwarmcos)
- `warmup_epochs` - Warmup epochs - `warmup_epochs` - Warmup epochs
- `max_epoch` - Maximum training epochs - `max_epoch` - Maximum training epochs
- `no_aug_epochs` - No augmentation epochs - `no_aug_epochs` - No augmentation epochs
- `min_lr_ratio` - Minimum LR ratio - `min_lr_ratio` - Minimum LR ratio
**Optimizer:** **Optimizer:**
- `momentum` - SGD momentum - `momentum` - SGD momentum
- `weight_decay` - Weight decay - `weight_decay` - Weight decay
**Augmentation:** **Augmentation:**
- `mosaic_prob` - Mosaic probability - `mosaic_prob` - Mosaic probability
- `mixup_prob` - Mixup probability - `mixup_prob` - Mixup probability
- `hsv_prob` - HSV augmentation probability - `hsv_prob` - HSV augmentation probability
- `flip_prob` - Flip probability - `flip_prob` - Flip probability
- `degrees` - Rotation degrees - `degrees` - Rotation degrees
- `translate` - Translation - `translate` - Translation
- `shear` - Shear - `shear` - Shear
- `mosaic_scale` - Mosaic scale range - `mosaic_scale` - Mosaic scale range
- `mixup_scale` - Mixup scale range - `mixup_scale` - Mixup scale range
- `enable_mixup` - Enable mixup - `enable_mixup` - Enable mixup
**Input/Output:** **Input/Output:**
- `input_size` - Training input size - `input_size` - Training input size
- `test_size` - Testing size - `test_size` - Testing size
- `random_size` - Random size range - `random_size` - Random size range
**Evaluation:** **Evaluation:**
- `eval_interval` - Evaluation interval - `eval_interval` - Evaluation interval
- `print_interval` - Print interval - `print_interval` - Print interval
## Customizing Base Configurations ## Customizing Base Configurations
### Adding a New Model ### Adding a New Model
Create a new file `data/yolox_MODELNAME.py`: Create a new file `data/yolox_MODELNAME.py`:
```python ```python
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-MODELNAME # Base configuration for YOLOX-MODELNAME
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-MODELNAME""" """Base experiment configuration for YOLOX-MODELNAME"""
# Define protected parameters # Define protected parameters
depth = 1.0 depth = 1.0
width = 1.0 width = 1.0
# ... other parameters # ... other parameters
``` ```
### Modifying Parameters ### Modifying Parameters
Edit the corresponding `yolox_*.py` file and update the `BaseExp` class attributes. Edit the corresponding `yolox_*.py` file and update the `BaseExp` class attributes.
**Example:** To change YOLOX-S max epochs: **Example:** To change YOLOX-S max epochs:
```python ```python
# In data/yolox_s.py # In data/yolox_s.py
class BaseExp: class BaseExp:
max_epoch = 500 # Changed from 300 max_epoch = 500 # Changed from 300
# ... other parameters # ... other parameters
``` ```
## Parameter Priority ## Parameter Priority
The merge logic follows this priority (highest to lowest): The merge logic follows this priority (highest to lowest):
1. **User form values** (if explicitly set, not None) 1. **User form values** (if explicitly set, not None)
2. **Base config values** (if transfer_learning='coco') 2. **Base config values** (if transfer_learning='coco')
3. **Default fallbacks** (hardcoded minimums) 3. **Default fallbacks** (hardcoded minimums)
## Example ## Example
### COCO Transfer Learning ### COCO Transfer Learning
``` ```
User sets in form: max_epoch=100, depth=0.5 User sets in form: max_epoch=100, depth=0.5
Base config (yolox_s.py) has: depth=0.33, width=0.50, max_epoch=300 Base config (yolox_s.py) has: depth=0.33, width=0.50, max_epoch=300
Result: depth=0.5 (user override), width=0.50 (base), max_epoch=100 (user override) Result: depth=0.5 (user override), width=0.50 (base), max_epoch=100 (user override)
``` ```
### Sketch Training ### Sketch Training
``` ```
User sets in form: max_epoch=100, depth=0.5 User sets in form: max_epoch=100, depth=0.5
No base config loaded No base config loaded
Result: depth=0.5 (user), max_epoch=100 (user), width=1.0 (default fallback) Result: depth=0.5 (user), max_epoch=100 (user), width=1.0 (default fallback)
``` ```
## Debugging ## Debugging
To see which base config was loaded, check Flask logs: To see which base config was loaded, check Flask logs:
``` ```
Loaded base config for yolox-s: ['depth', 'width', 'activation', ...] Loaded base config for yolox-s: ['depth', 'width', 'activation', ...]
``` ```
If base config fails to load: If base config fails to load:
``` ```
Warning: Could not load base config for yolox-s: [error message] Warning: Could not load base config for yolox-s: [error message]
Falling back to custom settings only Falling back to custom settings only
``` ```

2
backend/data/__init__.py Normal file → Executable file
View File

@@ -1 +1 @@
# Base experiment configurations for YOLOX models # Base experiment configurations for YOLOX models

158
backend/data/test_base_configs.py Normal file → Executable file
View File

@@ -1,79 +1,79 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Test script to demonstrate base configuration loading for YOLOX models Test script to demonstrate base configuration loading for YOLOX models
""" """
import sys import sys
import os import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..')) sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
from services.generate_yolox_exp import load_base_config from services.generate_yolox_exp import load_base_config
def test_base_configs(): def test_base_configs():
"""Test loading all base configurations""" """Test loading all base configurations"""
models = ['yolox-s', 'yolox-m', 'yolox-l', 'yolox-x'] models = ['yolox-s', 'yolox-m', 'yolox-l', 'yolox-x']
print("=" * 80) print("=" * 80)
print("YOLOX Base Configuration Test") print("YOLOX Base Configuration Test")
print("=" * 80) print("=" * 80)
for model in models: for model in models:
print(f"\n{'='*80}") print(f"\n{'='*80}")
print(f"Model: {model.upper()}") print(f"Model: {model.upper()}")
print(f"{'='*80}") print(f"{'='*80}")
try: try:
config = load_base_config(model) config = load_base_config(model)
# Group parameters by category # Group parameters by category
arch_params = ['depth', 'width', 'activation'] arch_params = ['depth', 'width', 'activation']
training_params = ['max_epoch', 'warmup_epochs', 'basic_lr_per_img', 'scheduler', training_params = ['max_epoch', 'warmup_epochs', 'basic_lr_per_img', 'scheduler',
'no_aug_epochs', 'min_lr_ratio'] 'no_aug_epochs', 'min_lr_ratio']
optimizer_params = ['momentum', 'weight_decay'] optimizer_params = ['momentum', 'weight_decay']
augmentation_params = ['mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', augmentation_params = ['mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob',
'degrees', 'translate', 'shear', 'mosaic_scale', 'degrees', 'translate', 'shear', 'mosaic_scale',
'mixup_scale', 'enable_mixup'] 'mixup_scale', 'enable_mixup']
input_params = ['input_size', 'test_size', 'random_size'] input_params = ['input_size', 'test_size', 'random_size']
eval_params = ['eval_interval', 'print_interval'] eval_params = ['eval_interval', 'print_interval']
print("\n[Architecture]") print("\n[Architecture]")
for param in arch_params: for param in arch_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Training Hyperparameters]") print("\n[Training Hyperparameters]")
for param in training_params: for param in training_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Optimizer]") print("\n[Optimizer]")
for param in optimizer_params: for param in optimizer_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Data Augmentation]") print("\n[Data Augmentation]")
for param in augmentation_params: for param in augmentation_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Input/Output]") print("\n[Input/Output]")
for param in input_params: for param in input_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print("\n[Evaluation]") print("\n[Evaluation]")
for param in eval_params: for param in eval_params:
if param in config: if param in config:
print(f" {param:25s} = {config[param]}") print(f" {param:25s} = {config[param]}")
print(f"\n✓ Successfully loaded {len(config)} parameters") print(f"\n✓ Successfully loaded {len(config)} parameters")
except Exception as e: except Exception as e:
print(f"✗ Error loading config: {e}") print(f"✗ Error loading config: {e}")
print("\n" + "="*80) print("\n" + "="*80)
print("Test Complete") print("Test Complete")
print("="*80) print("="*80)
if __name__ == '__main__': if __name__ == '__main__':
test_base_configs() test_base_configs()

30
backend/data/yolox_l.py Normal file → Executable file
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-L model # Base configuration for YOLOX-L model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-L""" """Base experiment configuration for YOLOX-L"""
# Model architecture (protected - always use these for yolox-l) # Model architecture (protected - always use these for yolox-l)
depth = 1.0 depth = 1.0
width = 1.0 width = 1.0
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

28
backend/data/yolox_m.py Normal file → Executable file
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-M model # Base configuration for YOLOX-M model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-M""" """Base experiment configuration for YOLOX-M"""
# Model architecture (protected - always use these for yolox-m) # Model architecture (protected - always use these for yolox-m)
depth = 0.67 depth = 0.67
width = 0.75 width = 0.75
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

34
backend/data/yolox_s.py Normal file → Executable file
View File

@@ -1,17 +1,17 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-S model # Base configuration for YOLOX-S model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-S""" """Base experiment configuration for YOLOX-S"""
# Model architecture (protected - always use these for yolox-s) # Model architecture (protected - always use these for yolox-s)
depth = 0.33 depth = 0.33
width = 0.50 width = 0.50
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

28
backend/data/yolox_x.py Normal file → Executable file
View File

@@ -1,15 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# Base configuration for YOLOX-X model # Base configuration for YOLOX-X model
# These parameters are preserved during transfer learning from COCO # These parameters are preserved during transfer learning from COCO
class BaseExp: class BaseExp:
"""Base experiment configuration for YOLOX-X""" """Base experiment configuration for YOLOX-X"""
# Model architecture (protected - always use these for yolox-x) # Model architecture (protected - always use these for yolox-x)
depth = 1.33 depth = 1.33
width = 1.25 width = 1.25
scheduler = "yoloxwarmcos" scheduler = "yoloxwarmcos"
activation = "silu" activation = "silu"

8
backend/database/__init__.py Normal file → Executable file
View File

@@ -1,4 +1,4 @@
# Database module # Database module
from database.database import db from database.database import db
__all__ = ['db'] __all__ = ['db']

10
backend/database/database.js
View File

@@ -1,10 +0,0 @@
// database.js
const { Sequelize } = require('sequelize');
const sequelize = new Sequelize('myapp', 'root', 'root', {
host: 'localhost',
dialect: 'mysql',
logging: false,
});
module.exports = sequelize;

18
backend/database/database.py Normal file → Executable file
View File

@@ -1,9 +1,9 @@
from flask_sqlalchemy import SQLAlchemy from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy() db = SQLAlchemy()
def init_db(app): def init_db(app):
"""Initialize database with app context""" """Initialize database with app context"""
db.init_app(app) db.init_app(app)
with app.app_context(): with app.app_context():
db.create_all() db.create_all()

12
backend/database/migration_add_image_dimensions.sql
View File

@@ -1,12 +0,0 @@
-- Migration: Add width and height columns to image table
-- Date: 2025-11-27
USE myapp;
-- Add width and height columns to image table
ALTER TABLE `image`
ADD COLUMN `width` FLOAT NULL AFTER `image_path`,
ADD COLUMN `height` FLOAT NULL AFTER `width`;
-- Verify the changes
DESCRIBE `image`;

250
backend/database/myapp.sql
View File

File diff suppressed because one or more lines are too long

40
backend/models/Annotation.js
View File

@@ -1,40 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Annotation = sequelize.define('Annotation', {
annotation_id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
},
image_id: {
type: DataTypes.INTEGER,
allowNull: false,
},
x: {
type: DataTypes.FLOAT,
allowNull: false,
},
y: {
type: DataTypes.FLOAT,
allowNull: false,
},
height: {
type: DataTypes.FLOAT,
allowNull: false,
},
width: {
type: DataTypes.FLOAT,
allowNull: false,
},
Label: {
type: DataTypes.STRING,
allowNull: false,
},
}, {
tableName: 'annotation',
timestamps: false,
});
module.exports = Annotation;

46
backend/models/Annotation.py Normal file → Executable file
View File

@@ -1,23 +1,23 @@
from database.database import db from database.database import db
class Annotation(db.Model): class Annotation(db.Model):
__tablename__ = 'annotation' __tablename__ = 'annotation'
annotation_id = db.Column(db.Integer, primary_key=True, autoincrement=True) annotation_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_id = db.Column(db.Integer, nullable=False) image_id = db.Column(db.Integer, db.ForeignKey('image.image_id', ondelete='CASCADE'), nullable=False)
x = db.Column(db.Float, nullable=False) x = db.Column(db.Float, nullable=False)
y = db.Column(db.Float, nullable=False) y = db.Column(db.Float, nullable=False)
height = db.Column(db.Float, nullable=False) height = db.Column(db.Float, nullable=False)
width = db.Column(db.Float, nullable=False) width = db.Column(db.Float, nullable=False)
Label = db.Column(db.String(255), nullable=False) Label = db.Column(db.String(255), nullable=False)
def to_dict(self): def to_dict(self):
return { return {
'annotation_id': self.annotation_id, 'annotation_id': self.annotation_id,
'image_id': self.image_id, 'image_id': self.image_id,
'x': self.x, 'x': self.x,
'y': self.y, 'y': self.y,
'height': self.height, 'height': self.height,
'width': self.width, 'width': self.width,
'Label': self.Label 'Label': self.Label
} }

21
backend/models/AnnotationProjectMapping.py Executable file
View File

@@ -0,0 +1,21 @@
from database.database import db
class AnnotationProjectMapping(db.Model):
"""Mapping between training project details and label studio projects (3NF)"""
__tablename__ = 'annotation_project_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
# Unique constraint: each label studio project can only be mapped once per training project details
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', name='uq_annotation_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id
}

25
backend/models/ClassMapping.py Executable file
View File

@@ -0,0 +1,25 @@
from database.database import db
class ClassMapping(db.Model):
"""Class name mappings for training project details (3NF)"""
__tablename__ = 'class_mapping'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
label_studio_project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
source_class = db.Column(db.String(255), nullable=False)
target_class = db.Column(db.String(255), nullable=False)
# Unique constraint: each source class can only be mapped once per project details AND label studio project
__table_args__ = (
db.UniqueConstraint('project_details_id', 'label_studio_project_id', 'source_class', name='uq_class_mapping'),
)
def to_dict(self):
return {
'id': self.id,
'project_details_id': self.project_details_id,
'label_studio_project_id': self.label_studio_project_id,
'source_class': self.source_class,
'target_class': self.target_class
}

35
backend/models/Images.js
View File

@@ -1,35 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Image = sequelize.define('Image', {
image_id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
},
image_path: {
type: DataTypes.STRING,
allowNull: false,
},
project_id: {
type: DataTypes.INTEGER,
allowNull: false,
},
width: {
type: DataTypes.FLOAT,
allowNull: true,
},
height: {
type: DataTypes.FLOAT,
allowNull: true,
},
}, {
tableName: 'image',
timestamps: false,
});
module.exports = Image;

38
backend/models/Images.py Normal file → Executable file
View File

@@ -1,19 +1,19 @@
from database.database import db from database.database import db
class Image(db.Model): class Image(db.Model):
__tablename__ = 'image' __tablename__ = 'image'
image_id = db.Column(db.Integer, primary_key=True, autoincrement=True) image_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
image_path = db.Column(db.String(500), nullable=False) image_path = db.Column(db.String(500), nullable=False)
project_id = db.Column(db.Integer, nullable=False) project_id = db.Column(db.Integer, db.ForeignKey('label_studio_project.project_id', ondelete='CASCADE'), nullable=False)
width = db.Column(db.Float) width = db.Column(db.Integer)
height = db.Column(db.Float) height = db.Column(db.Integer)
def to_dict(self): def to_dict(self):
return { return {
'image_id': self.image_id, 'image_id': self.image_id,
'image_path': self.image_path, 'image_path': self.image_path,
'project_id': self.project_id, 'project_id': self.project_id,
'width': self.width, 'width': self.width,
'height': self.height 'height': self.height
} }

24
backend/models/LabelStudioProject.js
View File

@@ -1,24 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Label_studio_project = sequelize.define('LabelStudioProject', {
project_id: {
type: DataTypes.INTEGER,
primaryKey: true,
unique: true,
allowNull: false,
},
title:{
type: DataTypes.STRING,
allowNull: false,
}
}, {
tableName: 'label_studio_project',
timestamps: false,
});
module.exports = Label_studio_project;

26
backend/models/LabelStudioProject.py Normal file → Executable file
View File

@@ -1,13 +1,13 @@
from database.database import db from database.database import db
class LabelStudioProject(db.Model): class LabelStudioProject(db.Model):
__tablename__ = 'label_studio_project' __tablename__ = 'label_studio_project'
project_id = db.Column(db.Integer, primary_key=True, unique=True) project_id = db.Column(db.Integer, primary_key=True, unique=True)
title = db.Column(db.String(255), nullable=False) title = db.Column(db.String(255), nullable=False)
def to_dict(self): def to_dict(self):
return { return {
'project_id': self.project_id, 'project_id': self.project_id,
'title': self.title 'title': self.title
} }

23
backend/models/ProjectClass.py Executable file
View File

@@ -0,0 +1,23 @@
from database.database import db
class ProjectClass(db.Model):
"""Class definitions for training projects (3NF)"""
__tablename__ = 'project_class'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False)
class_name = db.Column(db.String(255), nullable=False)
display_order = db.Column(db.Integer, default=0)
# Unique constraint: one class name per project
__table_args__ = (
db.UniqueConstraint('project_id', 'class_name', name='uq_project_class'),
)
def to_dict(self):
return {
'id': self.id,
'project_id': self.project_id,
'class_name': self.class_name,
'display_order': self.display_order
}

21
backend/models/Settings.py Executable file
View File

@@ -0,0 +1,21 @@
from database.database import db
class Settings(db.Model):
__tablename__ = 'settings'
id = db.Column(db.Integer, primary_key=True)
key = db.Column(db.String(255), unique=True, nullable=False)
value = db.Column(db.Text, nullable=True)
description = db.Column(db.String(500), nullable=True)
created_at = db.Column(db.DateTime, server_default=db.func.now())
updated_at = db.Column(db.DateTime, server_default=db.func.now(), onupdate=db.func.now())
def to_dict(self):
return {
'id': self.id,
'key': self.key,
'value': self.value,
'description': self.description,
'created_at': self.created_at.isoformat() if self.created_at else None,
'updated_at': self.updated_at.isoformat() if self.updated_at else None
}

38
backend/models/TrainingProject.js
View File

@@ -1,38 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Training_Project = sequelize.define('LabelStudioProject', {
project_id: {
type: DataTypes.INTEGER,
primaryKey: true,
unique: true,
allowNull: false,
autoIncrement: true,
},
title:{
type: DataTypes.STRING,
allowNull: false,
},
description: {
type: DataTypes.STRING,
},
classes: {
type: DataTypes.JSON,
allowNull: false,
},
project_image: {
type: DataTypes.BLOB,
},
project_image_type: {
type: DataTypes.STRING,
allowNull: true,
}
}, {
tableName: 'training_project',
timestamps: false,
});
module.exports = Training_Project;

64
backend/models/TrainingProject.py Normal file → Executable file
View File

@@ -1,28 +1,36 @@
from database.database import db from database.database import db
class TrainingProject(db.Model): class TrainingProject(db.Model):
__tablename__ = 'training_project' __tablename__ = 'training_project'
project_id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True) project_id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
title = db.Column(db.String(255), nullable=False) title = db.Column(db.String(255), nullable=False)
description = db.Column(db.String(500)) description = db.Column(db.String(500))
classes = db.Column(db.JSON, nullable=False) project_image = db.Column(db.LargeBinary)
project_image = db.Column(db.LargeBinary) project_image_type = db.Column(db.String(100))
project_image_type = db.Column(db.String(100))
# Relationship to classes (3NF)
def to_dict(self): classes_relation = db.relationship('ProjectClass', backref='project', lazy=True, cascade='all, delete-orphan')
result = {
'project_id': self.project_id, def to_dict(self, include_classes=True):
'title': self.title, result = {
'description': self.description, 'project_id': self.project_id,
'classes': self.classes, 'title': self.title,
'project_image_type': self.project_image_type 'description': self.description,
} 'project_image_type': self.project_image_type
if self.project_image: }
import base64
base64_data = base64.b64encode(self.project_image).decode('utf-8') # Include classes as array for backwards compatibility
mime_type = self.project_image_type or 'image/png' if include_classes:
result['project_image'] = f'data:{mime_type};base64,{base64_data}' from models.ProjectClass import ProjectClass
else: classes = ProjectClass.query.filter_by(project_id=self.project_id).order_by(ProjectClass.display_order).all()
result['project_image'] = None result['classes'] = [c.class_name for c in classes]
return result
if self.project_image:
import base64
base64_data = base64.b64encode(self.project_image).decode('utf-8')
mime_type = self.project_image_type or 'image/png'
result['project_image'] = f'data:{mime_type};base64,{base64_data}'
else:
result['project_image'] = None
return result

33
backend/models/TrainingProjectDetails.js
View File

@@ -1,33 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const TrainingProjectDetails = sequelize.define('TrainingProjectDetails', {
id: {
type: DataTypes.INTEGER,
primaryKey: true,
autoIncrement: true,
unique: true,
},
project_id: {
type: DataTypes.INTEGER,
allowNull: false,
unique: true,
},
annotation_projects: {
type: DataTypes.JSON,
allowNull: false,
},
class_map: {
type: DataTypes.JSON,
allowNull: true,
},
description: {
type: DataTypes.JSON,
allowNull: true,
}
}, {
tableName: 'training_project_details',
timestamps: false,
});
module.exports = TrainingProjectDetails;

54
backend/models/TrainingProjectDetails.py Normal file → Executable file
View File

@@ -1,19 +1,35 @@
from database.database import db from database.database import db
class TrainingProjectDetails(db.Model): class TrainingProjectDetails(db.Model):
__tablename__ = 'training_project_details' __tablename__ = 'training_project_details'
id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True) id = db.Column(db.Integer, primary_key=True, unique=True, autoincrement=True)
project_id = db.Column(db.Integer, nullable=False, unique=True) project_id = db.Column(db.Integer, db.ForeignKey('training_project.project_id', ondelete='CASCADE'), nullable=False, unique=True)
annotation_projects = db.Column(db.JSON, nullable=False) description_text = db.Column(db.Text) # Renamed from 'description' JSON to plain text
class_map = db.Column(db.JSON)
description = db.Column(db.JSON) # Relationships (3NF)
annotation_mappings = db.relationship('AnnotationProjectMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
def to_dict(self): class_mappings = db.relationship('ClassMapping', backref='project_details', lazy=True, cascade='all, delete-orphan')
return {
'id': self.id, def to_dict(self, include_mappings=True):
'project_id': self.project_id, result = {
'annotation_projects': self.annotation_projects, 'id': self.id,
'class_map': self.class_map, 'project_id': self.project_id,
'description': self.description 'description': self.description_text
} }
# Include mappings for backwards compatibility
if include_mappings:
from models.AnnotationProjectMapping import AnnotationProjectMapping
from models.ClassMapping import ClassMapping
# Get annotation projects as array
mappings = AnnotationProjectMapping.query.filter_by(project_details_id=self.id).all()
result['annotation_projects'] = [m.label_studio_project_id for m in mappings]
# Get class map as dictionary (grouped by label_studio_project_id for backwards compatibility)
# Return format: {source: target} (flattened across all projects)
class_maps = ClassMapping.query.filter_by(project_details_id=self.id).all()
result['class_map'] = {cm.source_class: cm.target_class for cm in class_maps}
return result

25
backend/models/TrainingSize.py Executable file
View File

@@ -0,0 +1,25 @@
from database.database import db
class TrainingSize(db.Model):
"""Size configurations for training (3NF - replaces JSON arrays)"""
__tablename__ = 'training_size'
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
training_id = db.Column(db.Integer, db.ForeignKey('training.id', ondelete='CASCADE'), nullable=False)
size_type = db.Column(db.String(50), nullable=False) # 'input_size', 'test_size', 'mosaic_scale', 'mixup_scale'
value_order = db.Column(db.Integer, nullable=False, default=0) # Order in array (0=first, 1=second)
value = db.Column(db.Float, nullable=False)
# Composite key ensures proper ordering
__table_args__ = (
db.UniqueConstraint('training_id', 'size_type', 'value_order', name='uq_training_size'),
)
def to_dict(self):
return {
'id': self.id,
'training_id': self.training_id,
'size_type': self.size_type,
'value_order': self.value_order,
'value': self.value
}

42
backend/models/__init__.py Normal file → Executable file
View File

@@ -1,16 +1,26 @@
# Import all models to ensure they are registered with SQLAlchemy # Import all models to ensure they are registered with SQLAlchemy
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from models.training import Training from models.training import Training
from models.LabelStudioProject import LabelStudioProject from models.LabelStudioProject import LabelStudioProject
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
from models.Settings import Settings
__all__ = [ from models.ProjectClass import ProjectClass
'TrainingProject', from models.AnnotationProjectMapping import AnnotationProjectMapping
'TrainingProjectDetails', from models.ClassMapping import ClassMapping
'Training', from models.TrainingSize import TrainingSize
'LabelStudioProject',
'Image', __all__ = [
'Annotation' 'TrainingProject',
] 'TrainingProjectDetails',
'Training',
'LabelStudioProject',
'Image',
'Annotation',
'Settings',
'ProjectClass',
'AnnotationProjectMapping',
'ClassMapping',
'TrainingSize'
]

30
backend/models/index.js
View File

@@ -1,30 +0,0 @@
const LabelStudioProject = require('./LabelStudioProject.js');
const Annotation = require('./Annotation.js');
const Image = require('./Images.js');
const sequelize = require('../database/database.js');
const TrainingProjectDetails = require('./TrainingProjectDetails.js');
const TrainingProject = require('./TrainingProject.js');
const Training = require('./training.js');
const Project = LabelStudioProject;
const Img = Image;
const Ann = Annotation;
// Associations
Project.hasMany(Img, { foreignKey: 'project_id' });
Img.belongsTo(Project, { foreignKey: 'project_id' });
Img.hasMany(Ann, { foreignKey: 'image_id' });
Ann.belongsTo(Img, { foreignKey: 'image_id' });
// TrainingProjectDetails <-> TrainingProject
TrainingProjectDetails.belongsTo(TrainingProject, { foreignKey: 'project_id' });
TrainingProject.hasOne(TrainingProjectDetails, { foreignKey: 'project_id' });
// Training <-> TrainingProjectDetails
Training.belongsTo(TrainingProjectDetails, { foreignKey: 'project_details_id' });
TrainingProjectDetails.hasMany(Training, { foreignKey: 'project_details_id' });
module.exports = { Project, Img, Ann, TrainingProjectDetails, TrainingProject, Training };

140
backend/models/training.js
View File

@@ -1,140 +0,0 @@
const { DataTypes } = require('sequelize');
const sequelize = require('../database/database.js');
const Training = sequelize.define('training', {
id: {
type: DataTypes.INTEGER,
autoIncrement: true,
unique: true,
primaryKey: true
},
exp_name: {
type: DataTypes.STRING(255)
},
max_epoch: {
type: DataTypes.INTEGER
},
depth: {
type: DataTypes.FLOAT
},
width: {
type: DataTypes.FLOAT
},
activation: {
type: DataTypes.STRING(255)
},
warmup_epochs: {
type: DataTypes.INTEGER
},
warmup_lr: {
type: DataTypes.FLOAT
},
basic_lr_per_img: {
type: DataTypes.FLOAT
},
scheduler: {
type: DataTypes.STRING(255)
},
no_aug_epochs: {
type: DataTypes.INTEGER
},
min_lr_ratio: {
type: DataTypes.FLOAT
},
ema: {
type: DataTypes.BOOLEAN
},
weight_decay: {
type: DataTypes.FLOAT
},
momentum: {
type: DataTypes.FLOAT
},
input_size: {
type: DataTypes.JSON
},
print_interval: {
type: DataTypes.INTEGER
},
eval_interval: {
type: DataTypes.INTEGER
},
save_history_ckpt: {
type: DataTypes.BOOLEAN
},
test_size: {
type: DataTypes.JSON
},
test_conf: {
type: DataTypes.FLOAT
},
nms_thre: {
type: DataTypes.FLOAT
},
multiscale_range: {
type: DataTypes.INTEGER
},
enable_mixup: {
type: DataTypes.BOOLEAN
},
mosaic_prob: {
type: DataTypes.FLOAT
},
mixup_prob: {
type: DataTypes.FLOAT
},
hsv_prob: {
type: DataTypes.FLOAT
},
flip_prob: {
type: DataTypes.FLOAT
},
degrees: {
type: DataTypes.FLOAT
},
mosaic_scale: {
type: DataTypes.JSON
},
mixup_scale: {
type: DataTypes.JSON
},
translate: {
type: DataTypes.FLOAT
},
shear: {
type: DataTypes.FLOAT
},
training_name: {
type: DataTypes.STRING(255)
},
project_details_id: {
type: DataTypes.INTEGER,
allowNull: false
},
seed: {
type: DataTypes.INTEGER
},
train: {
type: DataTypes.INTEGER
},
valid: {
type: DataTypes.INTEGER
},
test: {
type: DataTypes.INTEGER
},
selected_model: {
type: DataTypes.STRING(255)
},
transfer_learning: {
type: DataTypes.STRING(255)
},
model_upload: {
type: DataTypes.BLOB
}
}, {
tableName: 'training',
timestamps: false
});
module.exports = Training;

201
backend/models/training.py Normal file → Executable file
View File

@@ -1,92 +1,109 @@
from database.database import db from database.database import db
class Training(db.Model): class Training(db.Model):
__tablename__ = 'training' __tablename__ = 'training'
id = db.Column(db.Integer, primary_key=True, autoincrement=True, unique=True) id = db.Column(db.Integer, primary_key=True, autoincrement=True, unique=True)
exp_name = db.Column(db.String(255)) exp_name = db.Column(db.String(255))
max_epoch = db.Column(db.Integer) max_epoch = db.Column(db.Integer)
depth = db.Column(db.Float) depth = db.Column(db.Float)
width = db.Column(db.Float) width = db.Column(db.Float)
activation = db.Column(db.String(255)) activation = db.Column(db.String(255))
warmup_epochs = db.Column(db.Integer) warmup_epochs = db.Column(db.Integer)
warmup_lr = db.Column(db.Float) warmup_lr = db.Column(db.Float)
basic_lr_per_img = db.Column(db.Float) basic_lr_per_img = db.Column(db.Float)
scheduler = db.Column(db.String(255)) scheduler = db.Column(db.String(255))
no_aug_epochs = db.Column(db.Integer) no_aug_epochs = db.Column(db.Integer)
min_lr_ratio = db.Column(db.Float) min_lr_ratio = db.Column(db.Float)
ema = db.Column(db.Boolean) ema = db.Column(db.Boolean)
weight_decay = db.Column(db.Float) weight_decay = db.Column(db.Float)
momentum = db.Column(db.Float) momentum = db.Column(db.Float)
input_size = db.Column(db.JSON) # input_size moved to TrainingSize table
print_interval = db.Column(db.Integer) print_interval = db.Column(db.Integer)
eval_interval = db.Column(db.Integer) eval_interval = db.Column(db.Integer)
save_history_ckpt = db.Column(db.Boolean) save_history_ckpt = db.Column(db.Boolean)
test_size = db.Column(db.JSON) # test_size moved to TrainingSize table
test_conf = db.Column(db.Float) test_conf = db.Column(db.Float)
nms_thre = db.Column(db.Float) nms_thre = db.Column(db.Float)
multiscale_range = db.Column(db.Integer) multiscale_range = db.Column(db.Integer)
enable_mixup = db.Column(db.Boolean) enable_mixup = db.Column(db.Boolean)
mosaic_prob = db.Column(db.Float) mosaic_prob = db.Column(db.Float)
mixup_prob = db.Column(db.Float) mixup_prob = db.Column(db.Float)
hsv_prob = db.Column(db.Float) hsv_prob = db.Column(db.Float)
flip_prob = db.Column(db.Float) flip_prob = db.Column(db.Float)
degrees = db.Column(db.Float) degrees = db.Column(db.Float)
mosaic_scale = db.Column(db.JSON) # mosaic_scale moved to TrainingSize table
mixup_scale = db.Column(db.JSON) # mixup_scale moved to TrainingSize table
translate = db.Column(db.Float) translate = db.Column(db.Float)
shear = db.Column(db.Float) shear = db.Column(db.Float)
training_name = db.Column(db.String(255)) training_name = db.Column(db.String(255))
project_details_id = db.Column(db.Integer, nullable=False) project_details_id = db.Column(db.Integer, db.ForeignKey('training_project_details.id', ondelete='CASCADE'), nullable=False)
seed = db.Column(db.Integer) seed = db.Column(db.Integer)
train = db.Column(db.Integer) train = db.Column(db.Integer)
valid = db.Column(db.Integer) valid = db.Column(db.Integer)
test = db.Column(db.Integer) test = db.Column(db.Integer)
selected_model = db.Column(db.String(255)) selected_model = db.Column(db.String(255))
transfer_learning = db.Column(db.String(255)) transfer_learning = db.Column(db.String(255))
model_upload = db.Column(db.LargeBinary) model_upload = db.Column(db.LargeBinary)
def to_dict(self): # Relationship to size configurations (3NF)
return { size_configs = db.relationship('TrainingSize', backref='training', lazy=True, cascade='all, delete-orphan')
'id': self.id,
'exp_name': self.exp_name, def to_dict(self, include_sizes=True):
'max_epoch': self.max_epoch, result = {
'depth': self.depth, 'id': self.id,
'width': self.width, 'exp_name': self.exp_name,
'activation': self.activation, 'max_epoch': self.max_epoch,
'warmup_epochs': self.warmup_epochs, 'depth': self.depth,
'warmup_lr': self.warmup_lr, 'width': self.width,
'basic_lr_per_img': self.basic_lr_per_img, 'activation': self.activation,
'scheduler': self.scheduler, 'warmup_epochs': self.warmup_epochs,
'no_aug_epochs': self.no_aug_epochs, 'warmup_lr': self.warmup_lr,
'min_lr_ratio': self.min_lr_ratio, 'basic_lr_per_img': self.basic_lr_per_img,
'ema': self.ema, 'scheduler': self.scheduler,
'weight_decay': self.weight_decay, 'no_aug_epochs': self.no_aug_epochs,
'momentum': self.momentum, 'min_lr_ratio': self.min_lr_ratio,
'input_size': self.input_size, 'ema': self.ema,
'print_interval': self.print_interval, 'weight_decay': self.weight_decay,
'eval_interval': self.eval_interval, 'momentum': self.momentum,
'save_history_ckpt': self.save_history_ckpt, 'print_interval': self.print_interval,
'test_size': self.test_size, 'eval_interval': self.eval_interval,
'test_conf': self.test_conf, 'save_history_ckpt': self.save_history_ckpt,
'nms_thre': self.nms_thre, 'test_conf': self.test_conf,
'multiscale_range': self.multiscale_range, 'nms_thre': self.nms_thre,
'enable_mixup': self.enable_mixup, 'multiscale_range': self.multiscale_range,
'mosaic_prob': self.mosaic_prob, 'enable_mixup': self.enable_mixup,
'mixup_prob': self.mixup_prob, 'mosaic_prob': self.mosaic_prob,
'hsv_prob': self.hsv_prob, 'mixup_prob': self.mixup_prob,
'flip_prob': self.flip_prob, 'hsv_prob': self.hsv_prob,
'degrees': self.degrees, 'flip_prob': self.flip_prob,
'mosaic_scale': self.mosaic_scale, 'degrees': self.degrees,
'mixup_scale': self.mixup_scale, 'translate': self.translate,
'translate': self.translate, 'shear': self.shear,
'shear': self.shear, 'training_name': self.training_name,
'training_name': self.training_name, 'project_details_id': self.project_details_id,
'project_details_id': self.project_details_id, 'seed': self.seed,
'seed': self.seed, 'train': self.train,
'train': self.train, 'valid': self.valid,
'valid': self.valid, 'test': self.test,
'test': self.test, 'selected_model': self.selected_model,
'selected_model': self.selected_model, 'transfer_learning': self.transfer_learning
'transfer_learning': self.transfer_learning }
}
# Include size arrays for backwards compatibility
if include_sizes:
from models.TrainingSize import TrainingSize
def get_size_array(size_type):
sizes = TrainingSize.query.filter_by(
training_id=self.id,
size_type=size_type
).order_by(TrainingSize.value_order).all()
return [s.value for s in sizes] if sizes else None
result['input_size'] = get_size_array('input_size')
result['test_size'] = get_size_array('test_size')
result['mosaic_scale'] = get_size_array('mosaic_scale')
result['mixup_scale'] = get_size_array('mixup_scale')
return result

0
backend/node Normal file → Executable file
View File

0
backend/package-lock.json generated Normal file → Executable file
View File

0
backend/package.json Normal file → Executable file
View File

16
backend/requirements.txt Normal file → Executable file
View File

@@ -1,8 +1,8 @@
Flask==3.0.0 Flask==3.0.0
Flask-CORS==4.0.0 Flask-CORS==4.0.0
Flask-SQLAlchemy==3.1.1 Flask-SQLAlchemy==3.1.1
SQLAlchemy==2.0.23 SQLAlchemy==2.0.23
PyMySQL==1.1.0 PyMySQL==1.1.0
python-dotenv==1.0.0 python-dotenv==1.0.0
requests==2.31.0 requests==2.31.0
Pillow==10.1.0 Pillow==10.1.0

2
backend/routes/__init__.py Normal file → Executable file
View File

@@ -1 +1 @@
# Routes module # Routes module

496
backend/routes/api.js
View File

@@ -1,496 +0,0 @@
const express = require('express');
const multer = require('multer');
const upload = multer();
const TrainingProject = require('../models/TrainingProject.js');
const LabelStudioProject = require('../models/LabelStudioProject.js')
const { seedLabelStudio, updateStatus } = require('../services/seed-label-studio.js');
const fs = require('fs');
const path = require('path');
const {generateTrainingJson} = require('../services/generate-json-yolox.js')
const router = express.Router();
// Ensure JSON bodies are parsed for all routes
router.use(express.json());
router.get('/seed', async (req, res) => {
const result = await seedLabelStudio();
res.json(result);
});
// Trigger generate-json-yolox.js
router.post('/generate-yolox-json', async (req, res) => {
const { project_id } = req.body;
if (!project_id) {
return res.status(400).json({ message: 'Missing project_id in request body' });
}
try {
// Generate COCO JSONs
// Find all TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) {
return res.status(404).json({ message: 'No TrainingProjectDetails found for project ' + project_id });
}
// For each details row, generate coco.jsons and exp.py in projectfolder/project_details_id
const Training = require('../models/training.js');
const { saveYoloxExp } = require('../services/generate-yolox-exp.js');
const TrainingProject = require('../models/TrainingProject.js');
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
for (const details of detailsRows) {
const detailsId = details.id;
await generateTrainingJson(detailsId);
const trainings = await Training.findAll({ where: { project_details_id: detailsId } });
if (trainings.length === 0) continue;
// For each training, save exp.py in projectfolder/project_details_id
const outDir = path.join(__dirname, '..', projectName, String(detailsId));
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
for (const training of trainings) {
const expFilePath = path.join(outDir, 'exp.py');
await saveYoloxExp(training.id, expFilePath);
}
}
// Find all trainings for this project
// ...existing code...
res.json({ message: 'YOLOX JSON and exp.py generated for project ' + project_id });
} catch (err) {
console.error('Error generating YOLOX JSON:', err);
res.status(500).json({ message: 'Failed to generate YOLOX JSON', error: err.message });
}
});
// Start YOLOX training
const { spawn } = require('child_process');
router.post('/start-yolox-training', async (req, res) => {
try {
const { project_id, training_id } = req.body;
// Get project name
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
// Look up training row by id or project_details_id
const Training = require('../models/training.js');
let trainingRow = await Training.findByPk(training_id);
if (!trainingRow) {
trainingRow = await Training.findOne({ where: { project_details_id: training_id } });
}
if (!trainingRow) {
return res.status(404).json({ error: `Training row not found for id or project_details_id ${training_id}` });
}
const project_details_id = trainingRow.project_details_id;
// Use the generated exp.py from the correct project folder
const outDir = path.join(__dirname, '..', projectName, String(project_details_id));
const yoloxMainDir = '/home/kitraining/Yolox/YOLOX-main';
const expSrc = path.join(outDir, 'exp.py');
if (!fs.existsSync(expSrc)) {
return res.status(500).json({ error: `exp.py not found at ${expSrc}` });
}
// Source venv and run YOLOX training in YOLOX-main folder
const yoloxVenv = '/home/kitraining/Yolox/yolox_venv/bin/activate';
// Determine model argument based on selected_model and transfer_learning
let modelArg = '';
let cmd = '';
if (
trainingRow.transfer_learning &&
typeof trainingRow.transfer_learning === 'string' &&
trainingRow.transfer_learning.toLowerCase() === 'coco'
) {
// If transfer_learning is 'coco', add -o and modelArg
modelArg = ` -c /home/kitraining/Yolox/YOLOX-main/pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else if (
trainingRow.selected_model &&
trainingRow.selected_model.toLowerCase() === 'coco' &&
(!trainingRow.transfer_learning || trainingRow.transfer_learning === false)
) {
// If selected_model is 'coco' and not transfer_learning, add modelArg only
modelArg = ` -c /pretrained/${trainingRow.selected_model}`;
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
} else {
// Default: no modelArg
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16' --cache`;
}
console.log(cmd)
const child = spawn(cmd, { shell: true, cwd: yoloxMainDir });
child.stdout.pipe(process.stdout);
child.stderr.pipe(process.stderr);
res.json({ message: 'Training started' });
} catch (err) {
res.status(500).json({ error: 'Failed to start training', details: err.message });
}
});
// Get YOLOX training log
router.get('/training-log', async (req, res) => {
try {
const { project_id, training_id } = req.query;
const trainingProject = await TrainingProject.findByPk(project_id);
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
const outDir = path.join(__dirname, '..', projectName, String(training_id));
const logPath = path.join(outDir, 'training.log');
if (!fs.existsSync(logPath)) {
return res.status(404).json({ error: 'Log not found' });
}
const logData = fs.readFileSync(logPath, 'utf8');
res.json({ log: logData });
} catch (err) {
res.status(500).json({ error: 'Failed to fetch log', details: err.message });
}
});
router.post('/training-projects', upload.single('project_image'), async (req, res) => {
try {
const { title, description } = req.body;
const classes = JSON.parse(req.body.classes);
const project_image = req.file ? req.file.buffer : null;
const project_image_type = req.file ? req.file.mimetype : null;
await TrainingProject.create({
title,
description,
classes,
project_image,
project_image_type
});
res.json({ message: 'Project created!' });
} catch (error) {
console.error('Error creating project:', error);
res.status(500).json({ message: 'Failed to create project', error: error.message });
}
});
router.get('/training-projects', async (req, res) => {
try {
const projects = await TrainingProject.findAll();
// Convert BLOB to base64 data URL for each project
const serialized = projects.map(project => {
const plain = project.get({ plain: true });
if (plain.project_image) {
const base64 = Buffer.from(plain.project_image).toString('base64');
const mimeType = plain.project_image_type || 'image/png';
plain.project_image = `data:${mimeType};base64,${base64}`;
}
return plain;
});
res.json(serialized);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
}
});
router.get('/update-status', async (req, res) => {
res.json(updateStatus)
})
router.get('/label-studio-projects', async (req, res) => {
try {
const LabelStudioProject = require('../models/LabelStudioProject.js');
const Image = require('../models/Images.js');
const Annotation = require('../models/Annotation.js');
const labelStudioProjects = await LabelStudioProject.findAll();
const projectsWithCounts = await Promise.all(labelStudioProjects.map(async project => {
const plain = project.get({ plain: true });
// Get all images for this project
const images = await Image.findAll({ where: { project_id: plain.project_id } });
let annotationCounts = {};
if (images.length > 0) {
const imageIds = images.map(img => img.image_id);
// Get all annotations for these images
const annotations = await Annotation.findAll({ where: { image_id: imageIds } });
// Count by label
for (const ann of annotations) {
const label = ann.Label;
annotationCounts[label] = (annotationCounts[label] || 0) + 1;
}
}
plain.annotationCounts = annotationCounts;
return plain;
}));
res.json(projectsWithCounts);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
}
})
// POST endpoint to create TrainingProjectDetails with all fields
router.post('/training-project-details', async (req, res) => {
try {
const {
project_id,
annotation_projects,
class_map,
description
} = req.body;
if (!project_id || !annotation_projects) {
return res.status(400).json({ message: 'Missing required fields' });
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const created = await TrainingProjectDetails.create({
project_id,
annotation_projects,
class_map: class_map || null,
description: description || null
});
res.json({ message: 'TrainingProjectDetails created', details: created });
} catch (error) {
res.status(500).json({ message: 'Failed to create TrainingProjectDetails', error: error.message });
}
});
// GET endpoint to fetch all TrainingProjectDetails
router.get('/training-project-details', async (req, res) => {
try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findAll();
res.json(details);
} catch (error) {
res.status(500).json({ message: 'Failed to fetch TrainingProjectDetails', error: error.message });
}
});
// PUT endpoint to update class_map and description in TrainingProjectDetails
router.put('/training-project-details', async (req, res) => {
try {
const { project_id, class_map, description } = req.body;
if (!project_id || !class_map || !description) {
return res.status(400).json({ message: 'Missing required fields' });
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id } });
if (!details) {
return res.status(404).json({ message: 'TrainingProjectDetails not found' });
}
details.class_map = class_map;
details.description = description;
await details.save();
res.json({ message: 'Class map and description updated', details });
} catch (error) {
res.status(500).json({ message: 'Failed to update class map or description', error: error.message });
}
});
// POST endpoint to receive YOLOX settings and save to DB (handles multipart/form-data)
router.post('/yolox-settings', upload.any(), async (req, res) => {
try {
const settings = req.body;
// Debug: Log all received fields and types
console.log('--- YOLOX settings received ---');
console.log('settings:', settings);
if (req.files && req.files.length > 0) {
console.log('Files received:', req.files.map(f => ({ fieldname: f.fieldname, originalname: f.originalname, size: f.size })));
}
// Declare requiredFields once
const requiredFields = ['project_details_id', 'exp_name', 'max_epoch', 'depth', 'width', 'activation', 'train', 'valid', 'test', 'selected_model', 'transfer_learning'];
// Log types of required fields
requiredFields.forEach(field => {
console.log(`Field '${field}': value='${settings[field]}', type='${typeof settings[field]}'`);
});
// Map select_model to selected_model if present
if (settings && settings.select_model && !settings.selected_model) {
settings.selected_model = settings.select_model;
delete settings.select_model;
}
// Lookup project_details_id from project_id
if (!settings.project_id || isNaN(Number(settings.project_id))) {
throw new Error('Missing or invalid project_id in request. Cannot assign training to a project.');
}
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
let details = await TrainingProjectDetails.findOne({ where: { project_id: settings.project_id } });
if (!details) {
details = await TrainingProjectDetails.create({
project_id: settings.project_id,
annotation_projects: [],
class_map: null,
description: null
});
}
settings.project_details_id = details.id;
// Map 'act' from frontend to 'activation' for DB
if (settings.act !== undefined) {
settings.activation = settings.act;
delete settings.act;
}
// Type conversion for DB compatibility
[
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr', 'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum', 'print_interval', 'eval_interval', 'test_conf', 'nmsthre', 'multiscale_range', 'degrees', 'translate', 'shear', 'train', 'valid', 'test'
].forEach(f => {
if (settings[f] !== undefined) settings[f] = Number(settings[f]);
});
// Improved boolean conversion
['ema', 'enable_mixup', 'save_history_ckpt'].forEach(f => {
if (settings[f] !== undefined) {
if (typeof settings[f] === 'string') {
settings[f] = settings[f].toLowerCase() === 'true';
} else {
settings[f] = Boolean(settings[f]);
}
}
});
// Improved array conversion
['mosaic_scale', 'mixup_scale', 'scale'].forEach(f => {
if (settings[f] && typeof settings[f] === 'string') {
settings[f] = settings[f]
.split(',')
.map(s => Number(s.trim()))
.filter(n => !isNaN(n));
}
});
// Trim all string fields
Object.keys(settings).forEach(f => {
if (typeof settings[f] === 'string') settings[f] = settings[f].trim();
});
// Set default for transfer_learning if missing
if (settings.transfer_learning === undefined) settings.transfer_learning = false;
// Convert empty string seed to null
if ('seed' in settings && (settings.seed === '' || settings.seed === undefined)) {
settings.seed = null;
}
// Validate required fields for training table
for (const field of requiredFields) {
if (settings[field] === undefined || settings[field] === null || settings[field] === '') {
console.error('Missing required field:', field, 'Value:', settings[field]);
throw new Error('Missing required field: ' + field);
}
}
console.log('Received YOLOX settings:', settings);
// Handle uploaded model file (ckpt_upload)
if (req.files && req.files.length > 0) {
const ckptFile = req.files.find(f => f.fieldname === 'ckpt_upload');
if (ckptFile) {
const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = ckptFile.originalname || `uploaded_model_${settings.project_id}.pth`;
const filePath = path.join(uploadDir, filename);
fs.writeFileSync(filePath, ckptFile.buffer);
settings.model_upload = filePath;
}
}
// Save settings to DB only (no file)
const { pushYoloxExpToDb } = require('../services/push-yolox-exp.js');
const training = await pushYoloxExpToDb(settings);
res.json({ message: 'YOLOX settings saved to DB', training });
} catch (error) {
console.error('Error in /api/yolox-settings:', error.stack || error);
res.status(500).json({ message: 'Failed to save YOLOX settings', error: error.message });
}
});
// POST endpoint to receive binary model file and save to disk (not DB)
router.post('/yolox-settings/upload', async (req, res) => {
try {
const projectId = req.query.project_id;
if (!projectId) return res.status(400).json({ message: 'Missing project_id in query' });
// Save file to disk
const uploadDir = path.join(__dirname, '..', 'uploads');
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
const filename = req.headers['x-upload-filename'] || `uploaded_model_${projectId}.pth`;
const filePath = path.join(uploadDir, filename);
const chunks = [];
req.on('data', chunk => chunks.push(chunk));
req.on('end', async () => {
const buffer = Buffer.concat(chunks);
fs.writeFile(filePath, buffer, async err => {
if (err) {
console.error('Error saving file:', err);
return res.status(500).json({ message: 'Failed to save model file', error: err.message });
}
// Update latest training row for this project with file path
try {
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
// Find details row for this project
const details = await TrainingProjectDetails.findOne({ where: { project_id: projectId } });
if (!details) return res.status(404).json({ message: 'No TrainingProjectDetails found for project_id' });
// Find latest training for this details row
const training = await Training.findOne({ where: { project_details_id: details.id }, order: [['createdAt', 'DESC']] });
if (!training) return res.status(404).json({ message: 'No training found for project_id' });
// Save file path to model_upload field
training.model_upload = filePath;
await training.save();
res.json({ message: 'Model file uploaded and saved to disk', filename, trainingId: training.id });
} catch (dbErr) {
console.error('Error updating training with file path:', dbErr);
res.status(500).json({ message: 'File saved but failed to update training row', error: dbErr.message });
}
});
});
} catch (error) {
console.error('Error in /api/yolox-settings/upload:', error.stack || error);
res.status(500).json({ message: 'Failed to upload model file', error: error.message });
}
});
// GET endpoint to fetch all trainings (optionally filtered by project_id)
router.get('/trainings', async (req, res) => {
try {
const project_id = req.query.project_id;
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
if (project_id) {
// Find all details rows for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
if (!detailsRows || detailsRows.length === 0) return res.json([]);
// Get all trainings linked to any details row for this project
const detailsIds = detailsRows.map(d => d.id);
const trainings = await Training.findAll({ where: { project_details_id: detailsIds } });
return res.json(trainings);
} else {
// Return all trainings if no project_id is specified
const trainings = await Training.findAll();
return res.json(trainings);
}
} catch (error) {
res.status(500).json({ message: 'Failed to fetch trainings', error: error.message });
}
});
// DELETE endpoint to remove a training by id
router.delete('/trainings/:id', async (req, res) => {
try {
const Training = require('../models/training.js');
const id = req.params.id;
const deleted = await Training.destroy({ where: { id } });
if (deleted) {
res.json({ message: 'Training deleted' });
} else {
res.status(404).json({ message: 'Training not found' });
}
} catch (error) {
res.status(500).json({ message: 'Failed to delete training', error: error.message });
}
});
// DELETE endpoint to remove a training project and all related entries
router.delete('/training-projects/:id', async (req, res) => {
try {
const projectId = req.params.id;
const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const Training = require('../models/training.js');
// Find details row(s) for this project
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id: projectId } });
const detailsIds = detailsRows.map(d => d.id);
// Delete all trainings linked to these details
if (detailsIds.length > 0) {
await Training.destroy({ where: { project_details_id: detailsIds } });
await TrainingProjectDetails.destroy({ where: { project_id: projectId } });
}
// Delete the project itself
const deleted = await TrainingProject.destroy({ where: { project_id: projectId } });
if (deleted) {
res.json({ message: 'Training project and all related entries deleted' });
} else {
res.status(404).json({ message: 'Training project not found' });
}
} catch (error) {
res.status(500).json({ message: 'Failed to delete training project', error: error.message });
}
});
module.exports = router;

1437
backend/routes/api.py Normal file → Executable file
View File

File diff suppressed because it is too large Load Diff

68
backend/server.js Normal file → Executable file
View File

@@ -1,34 +1,34 @@
const express = require('express'); const express = require('express');
const cors = require('cors'); const cors = require('cors');
const path = require('path'); const path = require('path');
const sequelize = require('./database/database'); const sequelize = require('./database/database');
const app = express(); const app = express();
app.use(express.json()); app.use(express.json());
const port = 3000; const port = 3000;
const apiRouter = require('./routes/api.js'); const apiRouter = require('./routes/api.js');
app.use('/api', apiRouter); app.use('/api', apiRouter);
app.use(cors()); app.use(cors());
app.use(express.json()); app.use(express.json());
app.use(express.static(path.join(__dirname, '..'))); app.use(express.static(path.join(__dirname, '..')));
// Initialize DB and start server // Initialize DB and start server
(async () => { (async () => {
try { try {
await sequelize.authenticate(); await sequelize.authenticate();
console.log('DB connection established.'); console.log('DB connection established.');
await sequelize.sync(); // Only if you want Sequelize to ensure schema matches await sequelize.sync(); // Only if you want Sequelize to ensure schema matches
app.listen(port, '0.0.0.0', () => app.listen(port, '0.0.0.0', () =>
console.log(`Server running at http://0.0.0.0:${port}`) console.log(`Server running at http://0.0.0.0:${port}`)
); );
} catch (err) { } catch (err) {
console.error('Failed to start:', err); console.error('Failed to start:', err);
} }
})(); })();

2
backend/services/__init__.py Normal file → Executable file
View File

@@ -1 +1 @@
# Services module # Services module

92
backend/services/fetch-labelstudio.js
View File

@@ -1,92 +0,0 @@
const API_URL = 'http://192.168.1.19:8080/api';
const API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f';
const fetch = require('node-fetch');
async function fetchLableStudioProject(projectid) {
// 1. Trigger export
const exportUrl = `${API_URL}/projects/${projectid}/export?exportType=JSON_MIN`;
const headers = { Authorization: `Token ${API_TOKEN}` };
let res = await fetch(exportUrl, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to trigger export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to trigger export: ${res.status} ${res.statusText}`);
}
let data = await res.json();
// If data is an array, it's ready
if (Array.isArray(data)) return data;
// If not, poll for the export file
let fileUrl = data.download_url || data.url || null;
let tries = 0;
while (!fileUrl && tries < 20) {
await new Promise(r => setTimeout(r, 2000));
res = await fetch(exportUrl, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to poll export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to poll export: ${res.status} ${res.statusText}`);
}
data = await res.json();
fileUrl = data.download_url || data.url || null;
tries++;
}
if (!fileUrl) throw new Error('Label Studio export did not become ready');
// 2. Download the export file
res = await fetch(fileUrl.startsWith('http') ? fileUrl : `${API_URL.replace('/api','')}${fileUrl}`, { headers });
if (!res.ok) {
let errorText = await res.text().catch(() => '');
console.error(`Failed to download export: ${res.status} ${res.statusText} - ${errorText}`);
throw new Error(`Failed to download export: ${res.status} ${res.statusText}`);
}
return await res.json();
}
async function fetchProjectIdsAndTitles() {
try {
const response = await fetch(`${API_URL}/projects/`, {
headers: {
'Authorization': `Token ${API_TOKEN}`,
'Content-Type': 'application/json'
}
});
if (!response.ok) {
let errorText = await response.text().catch(() => '');
console.error(`Failed to fetch projects: ${response.status} ${response.statusText} - ${errorText}`);
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
if (!data.results || !Array.isArray(data.results)) {
throw new Error('API response does not contain results array');
}
// Extract id and title from each project
const projects = data.results.map(project => ({
id: project.id,
title: project.title
}));
console.log(projects)
return projects;
} catch (error) {
console.error('Failed to fetch projects:', error);
return [];
}
}
module.exports = { fetchLableStudioProject, fetchProjectIdsAndTitles };
//getLableStudioProject(20)
//fetchProjectIdsAndTitles()

178
backend/services/fetch_labelstudio.py Normal file → Executable file
View File

@@ -1,85 +1,93 @@
import requests import requests
import time import time
from services.settings_service import get_setting
API_URL = 'http://192.168.1.19:8080/api'
API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f' def get_api_credentials():
"""Get Label Studio API credentials from settings"""
def fetch_label_studio_project(project_id): api_url = get_setting('labelstudio_api_url', 'http://192.168.1.19:8080/api')
"""Fetch Label Studio project annotations""" api_token = get_setting('labelstudio_api_token', 'c1cef980b7c73004f4ee880a42839313b863869f')
export_url = f'{API_URL}/projects/{project_id}/export?exportType=JSON_MIN' return api_url, api_token
headers = {'Authorization': f'Token {API_TOKEN}'}
def fetch_label_studio_project(project_id):
# Trigger export """Fetch Label Studio project annotations"""
res = requests.get(export_url, headers=headers) API_URL, API_TOKEN = get_api_credentials()
if not res.ok:
error_text = res.text if res.text else '' export_url = f'{API_URL}/projects/{project_id}/export?exportType=JSON_MIN'
print(f'Failed to trigger export: {res.status_code} {res.reason} - {error_text}') headers = {'Authorization': f'Token {API_TOKEN}'}
raise Exception(f'Failed to trigger export: {res.status_code} {res.reason}')
# Trigger export
data = res.json() res = requests.get(export_url, headers=headers)
if not res.ok:
# If data is an array, it's ready error_text = res.text if res.text else ''
if isinstance(data, list): print(f'Failed to trigger export: {res.status_code} {res.reason} - {error_text}')
return data raise Exception(f'Failed to trigger export: {res.status_code} {res.reason}')
# If not, poll for the export file data = res.json()
file_url = data.get('download_url') or data.get('url')
tries = 0 # If data is an array, it's ready
if isinstance(data, list):
while not file_url and tries < 20: return data
time.sleep(2)
res = requests.get(export_url, headers=headers) # If not, poll for the export file
if not res.ok: file_url = data.get('download_url') or data.get('url')
error_text = res.text if res.text else '' tries = 0
print(f'Failed to poll export: {res.status_code} {res.reason} - {error_text}')
raise Exception(f'Failed to poll export: {res.status_code} {res.reason}') while not file_url and tries < 20:
time.sleep(2)
data = res.json() res = requests.get(export_url, headers=headers)
file_url = data.get('download_url') or data.get('url') if not res.ok:
tries += 1 error_text = res.text if res.text else ''
print(f'Failed to poll export: {res.status_code} {res.reason} - {error_text}')
if not file_url: raise Exception(f'Failed to poll export: {res.status_code} {res.reason}')
raise Exception('Label Studio export did not become ready')
data = res.json()
# Download the export file file_url = data.get('download_url') or data.get('url')
full_url = file_url if file_url.startswith('http') else f"{API_URL.replace('/api', '')}{file_url}" tries += 1
res = requests.get(full_url, headers=headers)
if not res.ok: if not file_url:
error_text = res.text if res.text else '' raise Exception('Label Studio export did not become ready')
print(f'Failed to download export: {res.status_code} {res.reason} - {error_text}')
raise Exception(f'Failed to download export: {res.status_code} {res.reason}') # Download the export file
full_url = file_url if file_url.startswith('http') else f"{API_URL.replace('/api', '')}{file_url}"
return res.json() res = requests.get(full_url, headers=headers)
if not res.ok:
def fetch_project_ids_and_titles(): error_text = res.text if res.text else ''
"""Fetch all Label Studio project IDs and titles""" print(f'Failed to download export: {res.status_code} {res.reason} - {error_text}')
try: raise Exception(f'Failed to download export: {res.status_code} {res.reason}')
response = requests.get(
f'{API_URL}/projects/', return res.json()
headers={
'Authorization': f'Token {API_TOKEN}', def fetch_project_ids_and_titles():
'Content-Type': 'application/json' """Fetch all Label Studio project IDs and titles"""
} API_URL, API_TOKEN = get_api_credentials()
)
try:
if not response.ok: response = requests.get(
error_text = response.text if response.text else '' f'{API_URL}/projects/',
print(f'Failed to fetch projects: {response.status_code} {response.reason} - {error_text}') headers={
raise Exception(f'HTTP error! status: {response.status_code}') 'Authorization': f'Token {API_TOKEN}',
'Content-Type': 'application/json'
data = response.json() }
)
if 'results' not in data or not isinstance(data['results'], list):
raise Exception('API response does not contain results array') if not response.ok:
error_text = response.text if response.text else ''
# Extract id and title from each project print(f'Failed to fetch projects: {response.status_code} {response.reason} - {error_text}')
projects = [ raise Exception(f'HTTP error! status: {response.status_code}')
{'id': project['id'], 'title': project['title']}
for project in data['results'] data = response.json()
]
print(projects) if 'results' not in data or not isinstance(data['results'], list):
return projects raise Exception('API response does not contain results array')
except Exception as error: # Extract id and title from each project
print(f'Failed to fetch projects: {error}') projects = [
return [] {'id': project['id'], 'title': project['title']}
for project in data['results']
]
print(projects)
return projects
except Exception as error:
print(f'Failed to fetch projects: {error}')
return []

176
backend/services/generate-json-yolox.js
View File

@@ -1,176 +0,0 @@
const TrainingProject = require('../models/TrainingProject.js');
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js')
const LabelStudioProject = require('../models/LabelStudioProject.js')
const Annotation = require('../models/Annotation.js')
const Images = require('../models/Images.js')
const fs = require('fs');
async function generateTrainingJson(trainingId){
// trainingId is now project_details_id
const trainingProjectDetails = await TrainingProjectDetails.findByPk(trainingId);
if (!trainingProjectDetails) throw new Error('No TrainingProjectDetails found for project_details_id ' + trainingId);
const detailsObj = trainingProjectDetails.get({ plain: true });
// Get parent project for name
const trainingProject = await TrainingProject.findByPk(detailsObj.project_id);
// Get split percentages (assume they are stored as train_percent, valid_percent, test_percent)
const trainPercent = detailsObj.train_percent || 85;
const validPercent = detailsObj.valid_percent || 10;
const testPercent = detailsObj.test_percent || 5;
let cocoImages = [];
let cocoAnnotations = [];
let cocoCategories = [];
let categoryMap = {};
let categoryId = 0;
let imageid = 0;
let annotationid = 0;
for (const cls of detailsObj.class_map) {
const asgMap = [];
const listAsg = cls[1];
for(const asg of listAsg){
asgMap.push ({ original: asg[0], mapped: asg[1] });
// Build category list and mapping
if (asg[1] && !(asg[1] in categoryMap)) {
categoryMap[asg[1]] = categoryId;
cocoCategories.push({ id: categoryId, name: asg[1], supercategory: '' });
categoryId++;
}
}
const images = await Images.findAll({ where: { project_id: cls[0] } });
for(const image of images){
imageid += 1;
let fileName = image.image_path;
if (fileName.includes('%20')) {
fileName = fileName.replace(/%20/g, ' ');
}
if (fileName && fileName.startsWith('/data/local-files/?d=')) {
fileName = fileName.replace('/data/local-files/?d=', '');
fileName = fileName.replace('/home/kitraining/home/kitraining/', '');
}
if (fileName && fileName.startsWith('home/kitraining/To_Annotate/')) {
fileName = fileName.replace('home/kitraining/To_Annotate/','');
}
// Get annotations for this image
const annotations = await Annotation.findAll({ where: { image_id: image.image_id } });
// Use image.width and image.height from DB (populated from original_width/original_height)
cocoImages.push({
id: imageid,
file_name: fileName,
width: image.width || 0,
height: image.height || 0
});
for (const annotation of annotations) {
// Translate class name using asgMap
let mappedClass = annotation.Label;
for (const mapEntry of asgMap) {
if (annotation.Label === mapEntry.original) {
mappedClass = mapEntry.mapped;
break;
}
}
// Only add annotation if mappedClass is valid
if (mappedClass && mappedClass in categoryMap) {
annotationid += 1;
let area = 0;
if (annotation.width && annotation.height) {
area = annotation.width * annotation.height;
}
cocoAnnotations.push({
id: annotationid,
image_id: imageid,
category_id: categoryMap[mappedClass],
bbox: [annotation.x, annotation.y, annotation.width, annotation.height],
area: area,
iscrowd: annotation.iscrowd || 0
});
}
}
}
}
// Shuffle images for random split using seed
function seededRandom(seed) {
let x = Math.sin(seed++) * 10000;
return x - Math.floor(x);
}
function shuffle(array, seed) {
for (let i = array.length - 1; i > 0; i--) {
const j = Math.floor(seededRandom(seed + i) * (i + 1));
[array[i], array[j]] = [array[j], array[i]];
}
}
// Use seed from detailsObj if present, else default to 42
const splitSeed = detailsObj.seed !== undefined && detailsObj.seed !== null ? Number(detailsObj.seed) : 42;
shuffle(cocoImages, splitSeed);
// Split images
const totalImages = cocoImages.length;
const trainCount = Math.floor(totalImages * trainPercent / 100);
const validCount = Math.floor(totalImages * validPercent / 100);
const testCount = totalImages - trainCount - validCount;
const trainImages = cocoImages.slice(0, trainCount);
const validImages = cocoImages.slice(trainCount, trainCount + validCount);
const testImages = cocoImages.slice(trainCount + validCount);
// Helper to get image ids for each split
const trainImageIds = new Set(trainImages.map(img => img.id));
const validImageIds = new Set(validImages.map(img => img.id));
const testImageIds = new Set(testImages.map(img => img.id));
// Split annotations
const trainAnnotations = cocoAnnotations.filter(ann => trainImageIds.has(ann.image_id));
const validAnnotations = cocoAnnotations.filter(ann => validImageIds.has(ann.image_id));
const testAnnotations = cocoAnnotations.filter(ann => testImageIds.has(ann.image_id));
// Build final COCO JSONs with info section
const buildCocoJson = (images, annotations, categories) => ({
images,
annotations,
categories
});
// Build COCO JSONs with info section
const trainJson = buildCocoJson(trainImages, trainAnnotations, cocoCategories);
const validJson = buildCocoJson(validImages, validAnnotations, cocoCategories);
const testJson = buildCocoJson(testImages, testAnnotations, cocoCategories);
// Create output directory: projectname/trainingid/annotations
const projectName = trainingProject && trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${detailsObj.project_id}`;
const outDir = `${projectName}/${trainingId}`;
const annotationsDir = `/home/kitraining/To_Annotate/annotations`;
if (!fs.existsSync(annotationsDir)) {
fs.mkdirSync(annotationsDir, { recursive: true });
}
// Write to files in the annotations directory
const trainPath = `${annotationsDir}/coco_project_${trainingId}_train.json`;
const validPath = `${annotationsDir}/coco_project_${trainingId}_valid.json`;
const testPath = `${annotationsDir}/coco_project_${trainingId}_test.json`;
fs.writeFileSync(trainPath, JSON.stringify(trainJson, null, 2));
fs.writeFileSync(validPath, JSON.stringify(validJson, null, 2));
fs.writeFileSync(testPath, JSON.stringify(testJson, null, 2));
console.log(`COCO JSON splits written to ${annotationsDir} for trainingId ${trainingId}`);
// Also generate inference exp.py in the same output directory as exp.py (project folder in workspace)
const { generateYoloxInferenceExp } = require('./generate-yolox-exp');
const path = require('path');
const projectFolder = path.join(__dirname, '..', projectName, String(trainingId));
if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true });
}
const inferenceExpPath = path.join(projectFolder, 'exp_infer.py');
generateYoloxInferenceExp(trainingId).then(expContent => {
fs.writeFileSync(inferenceExpPath, expContent);
console.log(`Inference exp.py written to ${inferenceExpPath}`);
}).catch(err => {
console.error('Failed to generate inference exp.py:', err);
});
}
module.exports = {generateTrainingJson};

135
backend/services/generate-yolox-exp.js
View File

@@ -1,135 +0,0 @@
const fs = require('fs');
const path = require('path');
const Training = require('../models/training.js');
const TrainingProject = require('../models/TrainingProject.js');
// Remove Python comments and legacy code
const exp_names = [
'YOLOX-s',
'YOLOX-m',
'YOLOX-l',
'YOLOX-x',
'YOLOX-Darknet53', //todo
'YOLOX-Nano',
'YOLOX-Tiny'
]
//TODO: Clean up generation of exp_names.py and remove second exp creation!!!
// Refactored: Accept trainingId, fetch info from DB
async function generateYoloxExp(trainingId) {
// Fetch training row from DB by project_details_id if not found by PK
let training = await Training.findByPk(trainingId);
if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } });
}
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// If transfer_learning is 'coco', just return the path to the default exp.py
if (training.transfer_learning === 'coco') {
const selectedModel = training.selected_model.toLowerCase().replace('-', '_');
const expSourcePath = `/home/kitraining/Yolox/YOLOX-main/exps/default/${selectedModel}.py`;
if (!fs.existsSync(expSourcePath)) {
throw new Error(`Default exp.py not found for model: ${selectedModel} at ${expSourcePath}`);
}
// Copy to project folder (e.g., /home/kitraining/coco_tool/backend/project_XX/YY/exp.py)
const projectDetailsId = training.project_details_id;
const projectFolder = path.resolve(__dirname, `../project_23/${projectDetailsId}`);
if (!fs.existsSync(projectFolder)) {
fs.mkdirSync(projectFolder, { recursive: true });
}
const expDestPath = path.join(projectFolder, 'exp.py');
fs.copyFileSync(expSourcePath, expDestPath);
return { type: 'default', expPath: expDestPath };
}
// If transfer_learning is 'sketch', generate a custom exp.py as before
if (training.transfer_learning === 'sketch') {
// ...existing custom exp.py generation logic here (copy from previous implementation)...
// For brevity, you can call generateYoloxInferenceExp or similar here, or inline the logic.
// Example:
const expContent = await generateYoloxInferenceExp(trainingId);
return { type: 'custom', expContent };
}
throw new Error('Unknown transfer_learning type: ' + training.transfer_learning);
}
async function saveYoloxExp(trainingId, outPath) {
const expResult = await generateYoloxExp(trainingId);
if (expResult.type === 'custom' && expResult.expContent) {
fs.writeFileSync(outPath, expResult.expContent);
return outPath;
} else if (expResult.type === 'default' && expResult.expPath) {
// Optionally copy the file if outPath is different
if (expResult.expPath !== outPath) {
fs.copyFileSync(expResult.expPath, outPath);
}
return outPath;
} else {
throw new Error('Unknown expResult type or missing content');
}
}
// Generate a second exp.py for inference, using the provided template and DB values
async function generateYoloxInferenceExp(trainingId, options = {}) {
let training = await Training.findByPk(trainingId);
if (!training) {
training = await Training.findOne({ where: { project_details_id: trainingId } });
}
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
// Always use the trainingId (project_details_id) for annotation file names
const projectDetailsId = training.project_details_id;
const dataDir = options.data_dir || '/home/kitraining/To_Annotate/';
const trainAnn = options.train_ann || `coco_project_${trainingId}_train.json`;
const valAnn = options.val_ann || `coco_project_${trainingId}_valid.json`;
const testAnn = options.test_ann || `coco_project_${trainingId}_test.json`;
// Get num_classes from TrainingProject.classes JSON
let numClasses = 80;
try {
const trainingProject = await TrainingProject.findByPk(projectDetailsId);
if (trainingProject && trainingProject.classes) {
let classesArr = trainingProject.classes;
if (typeof classesArr === 'string') {
classesArr = JSON.parse(classesArr);
}
if (Array.isArray(classesArr)) {
numClasses = classesArr.filter(c => c !== null && c !== undefined && c !== '').length;
} else if (typeof classesArr === 'object' && classesArr !== null) {
numClasses = Object.keys(classesArr).filter(k => classesArr[k] !== null && classesArr[k] !== undefined && classesArr[k] !== '').length;
}
}
} catch (e) {
console.warn('Could not determine num_classes from TrainingProject.classes:', e);
}
const depth = options.depth || training.depth || 1.00;
const width = options.width || training.width || 1.00;
const inputSize = options.input_size || training.input_size || [640, 640];
const mosaicScale = options.mosaic_scale || training.mosaic_scale || [0.1, 2];
const randomSize = options.random_size || training.random_size || [10, 20];
const testSize = options.test_size || training.test_size || [640, 640];
const expName = options.exp_name || 'inference_exp';
const enableMixup = options.enable_mixup !== undefined ? options.enable_mixup : false;
let expContent = '';
expContent += `#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.data_dir = "${dataDir}"\n self.train_ann = "${trainAnn}"\n self.val_ann = "${valAnn}"\n self.test_ann = "coco_project_${trainingId}_test.json"\n self.num_classes = ${numClasses}\n`;
// Set pretrained_ckpt if transfer_learning is 'coco'
if (training.transfer_learning && typeof training.transfer_learning === 'string' && training.transfer_learning.toLowerCase() === 'coco') {
const yoloxBaseDir = '/home/kitraining/Yolox/YOLOX-main';
const selectedModel = training.selected_model ? training.selected_model.replace(/\.pth$/i, '') : '';
if (selectedModel) {
expContent += ` self.pretrained_ckpt = r'${yoloxBaseDir}/pretrained/${selectedModel}.pth'\n`;
}
}
expContent += ` self.depth = ${depth}\n self.width = ${width}\n self.input_size = (${Array.isArray(inputSize) ? inputSize.join(', ') : inputSize})\n self.mosaic_scale = (${Array.isArray(mosaicScale) ? mosaicScale.join(', ') : mosaicScale})\n self.random_size = (${Array.isArray(randomSize) ? randomSize.join(', ') : randomSize})\n self.test_size = (${Array.isArray(testSize) ? testSize.join(', ') : testSize})\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]\n self.enable_mixup = ${enableMixup ? 'True' : 'False'}\n`;
return expContent;
}
// Save inference exp.py to a custom path
async function saveYoloxInferenceExp(trainingId, outPath, options = {}) {
const expContent = await generateYoloxInferenceExp(trainingId, options);
fs.writeFileSync(outPath, expContent);
return outPath;
}
module.exports = { generateYoloxExp, saveYoloxExp, generateYoloxInferenceExp, saveYoloxInferenceExp };

475
backend/services/generate_json_yolox.py Normal file → Executable file
View File

@@ -1,179 +1,296 @@
import json import json
import os import os
import math import math
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
def generate_training_json(training_id): def generate_training_json(training_id):
"""Generate COCO JSON for training, validation, and test sets""" """Generate COCO JSON for training, validation, and test sets
# training_id is now project_details_id
training_project_details = TrainingProjectDetails.query.get(training_id) Args:
training_id: Can be either a Training.id or TrainingProjectDetails.id
if not training_project_details: Function will automatically detect which one and find the correct details_id
raise Exception(f'No TrainingProjectDetails found for project_details_id {training_id}') """
from models.training import Training
details_obj = training_project_details.to_dict()
# First, try to get as a Training record
# Get parent project for name training_record = Training.query.get(training_id)
training_project = TrainingProject.query.get(details_obj['project_id'])
if training_record:
# Get split percentages (default values if not set) # It's a Training.id - use its project_details_id
train_percent = details_obj.get('train_percent', 85) details_id = training_record.project_details_id
valid_percent = details_obj.get('valid_percent', 10) print(f'[generate_training_json] Using training_id={training_id}, mapped to project_details_id={details_id}')
test_percent = details_obj.get('test_percent', 5) else:
# Try as TrainingProjectDetails.id directly
coco_images = [] details_id = training_id
coco_annotations = [] print(f'[generate_training_json] Using training_id={training_id} as project_details_id directly')
coco_categories = []
category_map = {} training_project_details = TrainingProjectDetails.query.get(details_id)
category_id = 0
image_id = 0 if not training_project_details:
annotation_id = 0 raise Exception(f'No TrainingProjectDetails found for id {training_id} (details_id: {details_id})')
for cls in details_obj['class_map']: details_obj = training_project_details.to_dict()
asg_map = []
list_asg = cls[1] # Get parent project for name
training_project = TrainingProject.query.get(details_obj['project_id'])
for asg in list_asg:
asg_map.append({'original': asg[0], 'mapped': asg[1]}) # Get the data directory setting for image paths
# Build category list and mapping from services.settings_service import get_setting
if asg[1] and asg[1] not in category_map: data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
category_map[asg[1]] = category_id
coco_categories.append({'id': category_id, 'name': asg[1], 'supercategory': ''}) # Fix UNC path if it's missing the \\ prefix
category_id += 1 # Check if it looks like a UNC path without proper prefix (e.g., "192.168.1.19\...")
if data_dir and not data_dir.startswith('\\\\') and not data_dir.startswith('/'):
# Get images for this project # Check if it starts with an IP address pattern
images = Image.query.filter_by(project_id=cls[0]).all() import re
if re.match(r'^\d+\.\d+\.\d+\.\d+[/\\]', data_dir):
for image in images: data_dir = '\\\\' + data_dir
image_id += 1
file_name = image.image_path # Ensure data_dir ends with separator
if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
# Clean up file path data_dir += os.sep
if '%20' in file_name:
file_name = file_name.replace('%20', ' ') # Get split percentages (default values if not set)
if file_name and file_name.startswith('/data/local-files/?d='): train_percent = details_obj.get('train_percent', 85)
file_name = file_name.replace('/data/local-files/?d=', '') valid_percent = details_obj.get('valid_percent', 10)
file_name = file_name.replace('/home/kitraining/home/kitraining/', '') test_percent = details_obj.get('test_percent', 5)
if file_name and file_name.startswith('home/kitraining/To_Annotate/'):
file_name = file_name.replace('home/kitraining/To_Annotate/', '') coco_images = []
coco_annotations = []
# Get annotations for this image coco_categories = []
annotations = Annotation.query.filter_by(image_id=image.image_id).all() category_map = {}
category_id = 0
coco_images.append({ image_id = 0
'id': image_id, annotation_id = 0
'file_name': file_name,
'width': image.width or 0, # Build category list and mapping from class_map dictionary {source: target}
'height': image.height or 0 class_map = details_obj.get('class_map', {})
})
for source_class, target_class in class_map.items():
for annotation in annotations: if target_class and target_class not in category_map:
# Translate class name using asg_map category_map[target_class] = category_id
mapped_class = annotation.Label coco_categories.append({'id': category_id, 'name': target_class, 'supercategory': ''})
for map_entry in asg_map: category_id += 1
if annotation.Label == map_entry['original']:
mapped_class = map_entry['mapped'] # Get all annotation projects (Label Studio project IDs)
break annotation_projects = details_obj.get('annotation_projects', [])
# Only add annotation if mapped_class is valid # Get class mappings from database grouped by Label Studio project
if mapped_class and mapped_class in category_map: from models.ClassMapping import ClassMapping
annotation_id += 1 all_mappings = ClassMapping.query.filter_by(project_details_id=training_id).all()
area = 0
if annotation.width and annotation.height: # Group mappings by Label Studio project ID
area = annotation.width * annotation.height mappings_by_project = {}
for mapping in all_mappings:
coco_annotations.append({ ls_proj_id = mapping.label_studio_project_id
'id': annotation_id, if ls_proj_id not in mappings_by_project:
'image_id': image_id, mappings_by_project[ls_proj_id] = {}
'category_id': category_map[mapped_class], mappings_by_project[ls_proj_id][mapping.source_class] = mapping.target_class
'bbox': [annotation.x, annotation.y, annotation.width, annotation.height],
'area': area, # Also add target class to category map if not present
'iscrowd': 0 if mapping.target_class and mapping.target_class not in category_map:
}) category_map[mapping.target_class] = category_id
coco_categories.append({'id': category_id, 'name': mapping.target_class, 'supercategory': ''})
# Shuffle images for random split using seed category_id += 1
def seeded_random(seed):
x = math.sin(seed) * 10000 # Iterate through each annotation project to collect images and annotations
return x - math.floor(x) for ls_project_id in annotation_projects:
# Get images for this Label Studio project
def shuffle(array, seed): images = Image.query.filter_by(project_id=ls_project_id).all()
for i in range(len(array) - 1, 0, -1):
j = int(seeded_random(seed + i) * (i + 1)) for image in images:
array[i], array[j] = array[j], array[i] image_id += 1
file_name = image.image_path
# Use seed from details_obj if present, else default to 42
split_seed = details_obj.get('seed', 42) # Clean up file path from Label Studio format
if split_seed is not None: if '%20' in file_name:
split_seed = int(split_seed) file_name = file_name.replace('%20', ' ')
else: if file_name and file_name.startswith('/data/local-files/?d='):
split_seed = 42 file_name = file_name.replace('/data/local-files/?d=', '')
shuffle(coco_images, split_seed) # Remove any Label Studio prefixes but keep full path
# Common Label Studio patterns
# Split images prefixes_to_remove = [
total_images = len(coco_images) '//192.168.1.19/home/kitraining/To_Annotate/',
train_count = int(total_images * train_percent / 100) '192.168.1.19/home/kitraining/To_Annotate/',
valid_count = int(total_images * valid_percent / 100) '/home/kitraining/home/kitraining/',
test_count = total_images - train_count - valid_count 'home/kitraining/To_Annotate/',
'/home/kitraining/To_Annotate/',
train_images = coco_images[0:train_count] ]
valid_images = coco_images[train_count:train_count + valid_count]
test_images = coco_images[train_count + valid_count:] # Try each prefix
for prefix in prefixes_to_remove:
# Helper to get image ids for each split if file_name.startswith(prefix):
train_image_ids = {img['id'] for img in train_images} file_name = file_name[len(prefix):]
valid_image_ids = {img['id'] for img in valid_images} break
test_image_ids = {img['id'] for img in test_images}
# Construct ABSOLUTE path using data_dir
# Split annotations # Detect platform for proper path handling
train_annotations = [ann for ann in coco_annotations if ann['image_id'] in train_image_ids] import platform
valid_annotations = [ann for ann in coco_annotations if ann['image_id'] in valid_image_ids] is_windows = platform.system() == 'Windows'
test_annotations = [ann for ann in coco_annotations if ann['image_id'] in test_image_ids]
# Normalize data_dir and file_name based on platform
# Build final COCO JSONs if is_windows:
def build_coco_json(images, annotations, categories): # Windows: use backslashes
return { normalized_data_dir = data_dir.rstrip('/\\').replace('/', '\\')
'images': images, file_name = file_name.replace('/', '\\')
'annotations': annotations, else:
'categories': categories # Linux/Mac: use forward slashes
} normalized_data_dir = data_dir.rstrip('/\\').replace('\\', '/')
file_name = file_name.replace('\\', '/')
train_json = build_coco_json(train_images, train_annotations, coco_categories)
valid_json = build_coco_json(valid_images, valid_annotations, coco_categories) # Check if already absolute path
test_json = build_coco_json(test_images, test_annotations, coco_categories) is_absolute = False
if is_windows:
# Create output directory is_absolute = file_name.startswith('\\\\') or (len(file_name) > 1 and file_name[1] == ':')
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}' else:
annotations_dir = '/home/kitraining/To_Annotate/annotations' is_absolute = file_name.startswith('/')
os.makedirs(annotations_dir, exist_ok=True)
if not is_absolute:
# Write to files # It's a relative path, combine with data_dir
train_path = f'{annotations_dir}/coco_project_{training_id}_train.json' if is_windows and normalized_data_dir.startswith('\\\\'):
valid_path = f'{annotations_dir}/coco_project_{training_id}_valid.json' # Windows UNC path
test_path = f'{annotations_dir}/coco_project_{training_id}_test.json' file_name = normalized_data_dir + '\\' + file_name
else:
with open(train_path, 'w') as f: # Regular path (Windows or Linux)
json.dump(train_json, f, indent=2) file_name = os.path.join(normalized_data_dir, file_name)
with open(valid_path, 'w') as f:
json.dump(valid_json, f, indent=2) # Get annotations for this image
with open(test_path, 'w') as f: annotations = Annotation.query.filter_by(image_id=image.image_id).all()
json.dump(test_json, f, indent=2)
coco_images.append({
print(f'COCO JSON splits written to {annotations_dir} for trainingId {training_id}') 'id': image_id,
'file_name': file_name, # Use absolute path
# Also generate inference exp.py 'width': image.width or 0,
from services.generate_yolox_exp import generate_yolox_inference_exp 'height': image.height or 0
project_folder = os.path.join(os.path.dirname(__file__), '..', project_name, str(training_id)) })
os.makedirs(project_folder, exist_ok=True)
for annotation in annotations:
inference_exp_path = os.path.join(project_folder, 'exp_infer.py') # Translate class name using class_map for this specific Label Studio project
try: original_class = annotation.Label
exp_content = generate_yolox_inference_exp(training_id) project_class_map = mappings_by_project.get(ls_project_id, {})
with open(inference_exp_path, 'w') as f: mapped_class = project_class_map.get(original_class, original_class)
f.write(exp_content)
print(f'Inference exp.py written to {inference_exp_path}') # Only add annotation if mapped_class is valid
except Exception as err: if mapped_class and mapped_class in category_map:
print(f'Failed to generate inference exp.py: {err}') annotation_id += 1
area = 0
if annotation.width and annotation.height:
area = annotation.width * annotation.height
coco_annotations.append({
'id': annotation_id,
'image_id': image_id,
'category_id': category_map[mapped_class],
'bbox': [annotation.x, annotation.y, annotation.width, annotation.height],
'area': area,
'iscrowd': 0
})
# Shuffle images for random split using seed
def seeded_random(seed):
x = math.sin(seed) * 10000
return x - math.floor(x)
def shuffle(array, seed):
for i in range(len(array) - 1, 0, -1):
j = int(seeded_random(seed + i) * (i + 1))
array[i], array[j] = array[j], array[i]
# Use seed from details_obj if present, else default to 42
split_seed = details_obj.get('seed', 42)
if split_seed is not None:
split_seed = int(split_seed)
else:
split_seed = 42
shuffle(coco_images, split_seed)
# Split images
total_images = len(coco_images)
train_count = int(total_images * train_percent / 100)
valid_count = int(total_images * valid_percent / 100)
test_count = total_images - train_count - valid_count
train_images = coco_images[0:train_count]
valid_images = coco_images[train_count:train_count + valid_count]
test_images = coco_images[train_count + valid_count:]
# Helper to get image ids for each split
train_image_ids = {img['id'] for img in train_images}
valid_image_ids = {img['id'] for img in valid_images}
test_image_ids = {img['id'] for img in test_images}
# Split annotations
train_annotations = [ann for ann in coco_annotations if ann['image_id'] in train_image_ids]
valid_annotations = [ann for ann in coco_annotations if ann['image_id'] in valid_image_ids]
test_annotations = [ann for ann in coco_annotations if ann['image_id'] in test_image_ids]
# Build final COCO JSONs
def build_coco_json(images, annotations, categories):
return {
'images': images,
'annotations': annotations,
'categories': categories
}
train_json = build_coco_json(train_images, train_annotations, coco_categories)
valid_json = build_coco_json(valid_images, valid_annotations, coco_categories)
test_json = build_coco_json(test_images, test_annotations, coco_categories)
# Create output directory
from services.settings_service import get_setting
from models.training import Training
output_base_path = get_setting('yolox_output_path', './backend')
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}'
# Get training record to use its name and ID for folder and file names
# Use the same training_id that was passed in (if it was a Training.id)
# or find the first training for this details_id
if not training_record:
training_record = Training.query.filter_by(project_details_id=details_id).first()
if training_record:
training_folder_name = f"{training_record.exp_name or training_record.training_name or 'training'}_{training_record.id}"
training_folder_name = training_folder_name.replace(' ', '_')
training_file_id = training_record.id
else:
training_folder_name = str(details_id)
training_file_id = details_id
# Save annotations to the configured output folder
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
os.makedirs(annotations_dir, exist_ok=True)
# Write to files
train_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_train.json')
valid_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_valid.json')
test_path = os.path.join(annotations_dir, f'coco_project_{training_file_id}_test.json')
with open(train_path, 'w') as f:
json.dump(train_json, f, indent=2)
with open(valid_path, 'w') as f:
json.dump(valid_json, f, indent=2)
with open(test_path, 'w') as f:
json.dump(test_json, f, indent=2)
print(f'COCO JSON splits written to {annotations_dir} for training_id={training_file_id} (details_id={details_id})')
# Also generate inference exp.py
from services.generate_yolox_exp import generate_yolox_inference_exp
project_folder = os.path.join(output_base_path, project_name, str(training_id))
os.makedirs(project_folder, exist_ok=True)
inference_exp_path = os.path.join(project_folder, 'exp_infer.py')
try:
exp_content = generate_yolox_inference_exp(training_id)
with open(inference_exp_path, 'w') as f:
f.write(exp_content)
print(f'Inference exp.py written to {inference_exp_path}')
except Exception as err:
print(f'Failed to generate inference exp.py: {err}')

601
backend/services/generate_yolox_exp.py Normal file → Executable file
View File

@@ -1,228 +1,373 @@
import os import os
import shutil import shutil
import importlib.util import importlib.util
from models.training import Training from models.training import Training
from models.TrainingProject import TrainingProject from models.TrainingProject import TrainingProject
def load_base_config(selected_model): def load_base_config(selected_model):
"""Load base configuration for a specific YOLOX model""" """Load base configuration for a specific YOLOX model"""
model_name = selected_model.lower().replace('-', '_').replace('.pth', '') model_name = selected_model.lower().replace('-', '_').replace('.pth', '')
base_config_path = os.path.join(os.path.dirname(__file__), '..', 'data', f'{model_name}.py') base_config_path = os.path.join(os.path.dirname(__file__), '..', 'data', f'{model_name}.py')
if not os.path.exists(base_config_path): if not os.path.exists(base_config_path):
raise Exception(f'Base configuration not found for model: {model_name} at {base_config_path}') raise Exception(f'Base configuration not found for model: {model_name} at {base_config_path}')
# Load the module dynamically # Load the module dynamically
spec = importlib.util.spec_from_file_location(f"base_config_{model_name}", base_config_path) spec = importlib.util.spec_from_file_location(f"base_config_{model_name}", base_config_path)
module = importlib.util.module_from_spec(spec) module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module) spec.loader.exec_module(module)
# Extract all attributes from BaseExp class # Extract all attributes from BaseExp class
base_exp = module.BaseExp() base_exp = module.BaseExp()
base_config = {} base_config = {}
for attr in dir(base_exp): for attr in dir(base_exp):
if not attr.startswith('_'): if not attr.startswith('_'):
base_config[attr] = getattr(base_exp, attr) base_config[attr] = getattr(base_exp, attr)
return base_config return base_config
def generate_yolox_exp(training_id): def generate_yolox_exp(training_id):
"""Generate YOLOX exp.py file""" """Generate YOLOX exp.py file"""
# Fetch training row from DB # Fetch training row from DB
training = Training.query.get(training_id) training = Training.query.get(training_id)
if not training: if not training:
training = Training.query.filter_by(project_details_id=training_id).first() training = Training.query.filter_by(project_details_id=training_id).first()
if not training: if not training:
raise Exception(f'Training not found for trainingId or project_details_id: {training_id}') raise Exception(f'Training not found for trainingId or project_details_id: {training_id}')
# If transfer_learning is 'coco', generate exp using base config + custom settings # If transfer_learning is 'coco', generate exp using base config + custom settings
if training.transfer_learning == 'coco': if training.transfer_learning == 'coco':
exp_content = generate_yolox_inference_exp(training_id, use_base_config=True) exp_content = generate_yolox_inference_exp(training_id, use_base_config=True)
return {'type': 'custom', 'expContent': exp_content} return {'type': 'custom', 'expContent': exp_content}
# If transfer_learning is 'sketch', generate custom exp.py # If transfer_learning is 'sketch', generate custom exp.py
if training.transfer_learning == 'sketch': if training.transfer_learning == 'sketch':
exp_content = generate_yolox_inference_exp(training_id, use_base_config=False) exp_content = generate_yolox_inference_exp(training_id, use_base_config=False)
return {'type': 'custom', 'expContent': exp_content} return {'type': 'custom', 'expContent': exp_content}
raise Exception(f'Unknown transfer_learning type: {training.transfer_learning}') raise Exception(f'Unknown transfer_learning type: {training.transfer_learning}')
def save_yolox_exp(training_id, out_path): def save_yolox_exp(training_id, out_path):
"""Save YOLOX exp.py to specified path""" """Save YOLOX exp.py to specified path"""
exp_result = generate_yolox_exp(training_id) exp_result = generate_yolox_exp(training_id)
if exp_result['type'] == 'custom' and 'expContent' in exp_result: if exp_result['type'] == 'custom' and 'expContent' in exp_result:
with open(out_path, 'w') as f: with open(out_path, 'w') as f:
f.write(exp_result['expContent']) f.write(exp_result['expContent'])
return out_path return out_path
elif exp_result['type'] == 'default' and 'expPath' in exp_result: elif exp_result['type'] == 'default' and 'expPath' in exp_result:
# Optionally copy the file if outPath is different # Optionally copy the file if outPath is different
if exp_result['expPath'] != out_path: if exp_result['expPath'] != out_path:
shutil.copyfile(exp_result['expPath'], out_path) shutil.copyfile(exp_result['expPath'], out_path)
return out_path return out_path
else: else:
raise Exception('Unknown expResult type or missing content') raise Exception('Unknown expResult type or missing content')
def generate_yolox_inference_exp(training_id, options=None, use_base_config=False): def generate_yolox_inference_exp(training_id, options=None, use_base_config=False):
"""Generate inference exp.py using DB values """Generate inference exp.py using DB values
Args: Args:
training_id: The training/project_details ID training_id: The training/project_details ID
options: Optional overrides for data paths options: Optional overrides for data paths
use_base_config: If True, load base config and only override with user-defined values use_base_config: If True, load base config and only override with user-defined values
""" """
if options is None: if options is None:
options = {} options = {}
training = Training.query.get(training_id) training = Training.query.get(training_id)
if not training: if not training:
training = Training.query.filter_by(project_details_id=training_id).first() training = Training.query.filter_by(project_details_id=training_id).first()
if not training: if not training:
raise Exception(f'Training not found for trainingId or project_details_id: {training_id}') raise Exception(f'Training not found for trainingId or project_details_id: {training_id}')
# Always use the training_id (project_details_id) for annotation file names # Always use the project_details_id for annotation file names and paths
project_details_id = training.project_details_id project_details_id = training.project_details_id
data_dir = options.get('data_dir', '/home/kitraining/To_Annotate/') # Get annotation file names from options or use defaults
train_ann = options.get('train_ann', f'coco_project_{training_id}_train.json') # Use training.id (not project_details_id) for consistency with generate_training_json
val_ann = options.get('val_ann', f'coco_project_{training_id}_valid.json') train_ann = options.get('train_ann', f'coco_project_{training_id}_train.json')
test_ann = options.get('test_ann', f'coco_project_{training_id}_test.json') val_ann = options.get('val_ann', f'coco_project_{training_id}_valid.json')
test_ann = options.get('test_ann', f'coco_project_{training_id}_test.json')
# Get num_classes from TrainingProject.classes JSON
num_classes = 80 # Get data_dir - this should point to where IMAGES are located (not annotations)
try: # YOLOX will combine data_dir + file_name from COCO JSON to find images
training_project = TrainingProject.query.get(project_details_id) # The annotations are in a separate location (output folder)
if training_project and training_project.classes: from services.settings_service import get_setting
classes_arr = training_project.classes from models.TrainingProjectDetails import TrainingProjectDetails
if isinstance(classes_arr, str):
import json if 'data_dir' in options:
classes_arr = json.loads(classes_arr) data_dir = options['data_dir']
else:
if isinstance(classes_arr, list): # Use the yolox_data_dir setting - this is where training images are stored
num_classes = len([c for c in classes_arr if c not in [None, '']]) data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
elif isinstance(classes_arr, dict): # Ensure it ends with a separator
num_classes = len([k for k, v in classes_arr.items() if v not in [None, '']]) if not data_dir.endswith(os.sep) and not data_dir.endswith('/'):
except Exception as e: data_dir += os.sep
print(f'Could not determine num_classes from TrainingProject.classes: {e}')
# Get num_classes from ProjectClass table (3NF)
# Initialize config dictionary num_classes = 80
config = {} try:
from models.ProjectClass import ProjectClass
# If using base config (transfer learning from COCO), load protected parameters first training_project = TrainingProject.query.get(project_details_id)
if use_base_config and training.selected_model: if training_project:
try: # Count classes from ProjectClass table
base_config = load_base_config(training.selected_model) class_count = ProjectClass.query.filter_by(project_id=training_project.project_id).count()
config.update(base_config) if class_count > 0:
print(f'Loaded base config for {training.selected_model}: {list(base_config.keys())}') num_classes = class_count
except Exception as e: except Exception as e:
print(f'Warning: Could not load base config for {training.selected_model}: {e}') print(f'Could not determine num_classes from ProjectClass: {e}')
print('Falling back to custom settings only')
# Initialize config dictionary
# Override with user-defined values from training table (only if they exist and are not None) config = {}
user_overrides = {
'depth': training.depth, # If using base config (transfer learning from COCO), load protected parameters first
'width': training.width, if use_base_config and training.selected_model:
'input_size': training.input_size, try:
'mosaic_scale': training.mosaic_scale, base_config = load_base_config(training.selected_model)
'test_size': training.test_size, config.update(base_config)
'enable_mixup': training.enable_mixup, print(f'Loaded base config for {training.selected_model}: {list(base_config.keys())}')
'max_epoch': training.max_epoch, except Exception as e:
'warmup_epochs': training.warmup_epochs, print(f'Warning: Could not load base config for {training.selected_model}: {e}')
'warmup_lr': training.warmup_lr, print('Falling back to custom settings only')
'basic_lr_per_img': training.basic_lr_per_img,
'scheduler': training.scheduler, # Get size arrays from TrainingSize table (3NF)
'no_aug_epochs': training.no_aug_epochs, from models.TrainingSize import TrainingSize
'min_lr_ratio': training.min_lr_ratio,
'ema': training.ema, def get_size_array(training_id, size_type):
'weight_decay': training.weight_decay, """Helper to get size array from TrainingSize table"""
'momentum': training.momentum, sizes = TrainingSize.query.filter_by(
'print_interval': training.print_interval, training_id=training_id,
'eval_interval': training.eval_interval, size_type=size_type
'test_conf': training.test_conf, ).order_by(TrainingSize.value_order).all()
'nms_thre': training.nms_thre, return [s.value for s in sizes] if sizes else None
'mosaic_prob': training.mosaic_prob,
'mixup_prob': training.mixup_prob, input_size = get_size_array(training.id, 'input_size')
'hsv_prob': training.hsv_prob, test_size = get_size_array(training.id, 'test_size')
'flip_prob': training.flip_prob, mosaic_scale = get_size_array(training.id, 'mosaic_scale')
'degrees': training.degrees, mixup_scale = get_size_array(training.id, 'mixup_scale')
'translate': training.translate,
'shear': training.shear, # Override with user-defined values from training table (only if they exist and are not None)
'mixup_scale': training.mixup_scale, user_overrides = {
'activation': training.activation, 'depth': training.depth,
} 'width': training.width,
'input_size': input_size,
# Only override if value is explicitly set (not None) 'mosaic_scale': mosaic_scale,
for key, value in user_overrides.items(): 'test_size': test_size,
if value is not None: 'enable_mixup': training.enable_mixup,
config[key] = value 'max_epoch': training.max_epoch,
'warmup_epochs': training.warmup_epochs,
# Apply any additional options overrides 'warmup_lr': training.warmup_lr,
config.update(options) 'basic_lr_per_img': training.basic_lr_per_img,
'scheduler': training.scheduler,
# Set defaults for any missing required parameters 'no_aug_epochs': training.no_aug_epochs,
config.setdefault('depth', 1.00) 'min_lr_ratio': training.min_lr_ratio,
config.setdefault('width', 1.00) 'ema': training.ema,
config.setdefault('input_size', [640, 640]) 'weight_decay': training.weight_decay,
config.setdefault('mosaic_scale', [0.1, 2]) 'momentum': training.momentum,
config.setdefault('random_size', [10, 20]) 'print_interval': training.print_interval,
config.setdefault('test_size', [640, 640]) 'eval_interval': training.eval_interval,
config.setdefault('enable_mixup', False) 'test_conf': training.test_conf,
config.setdefault('exp_name', 'inference_exp') 'nms_thre': training.nms_thre,
'mosaic_prob': training.mosaic_prob,
# Build exp content 'mixup_prob': training.mixup_prob,
exp_content = f'''#!/usr/bin/env python3 'hsv_prob': training.hsv_prob,
# -*- coding:utf-8 -*- 'flip_prob': training.flip_prob,
# Copyright (c) Megvii, Inc. and its affiliates. 'degrees': training.degrees,
'translate': training.translate,
import os 'shear': training.shear,
'mixup_scale': mixup_scale,
from yolox.exp import Exp as MyExp 'activation': training.activation,
}
class Exp(MyExp): # Only override if value is explicitly set (not None)
def __init__(self): for key, value in user_overrides.items():
super(Exp, self).__init__() if value is not None:
self.data_dir = "{data_dir}" config[key] = value
self.train_ann = "{train_ann}"
self.val_ann = "{val_ann}" # Apply any additional options overrides
self.test_ann = "{test_ann}" config.update(options)
self.num_classes = {num_classes}
''' # Set defaults for any missing required parameters
config.setdefault('depth', 1.00)
# Set pretrained_ckpt if transfer_learning is 'coco' config.setdefault('width', 1.00)
if training.transfer_learning and isinstance(training.transfer_learning, str) and training.transfer_learning.lower() == 'coco': config.setdefault('input_size', [640, 640])
yolox_base_dir = '/home/kitraining/Yolox/YOLOX-main' config.setdefault('mosaic_scale', [0.1, 2])
selected_model = training.selected_model.replace('.pth', '') if training.selected_model else '' config.setdefault('random_size', [10, 20])
if selected_model: config.setdefault('test_size', [640, 640])
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n" config.setdefault('enable_mixup', False)
config.setdefault('exp_name', 'inference_exp')
# Format arrays
def format_value(val): # Prepare data_dir for template - escape backslashes and remove trailing separator
if isinstance(val, (list, tuple)): data_dir_clean = data_dir.rstrip('/\\')
return '(' + ', '.join(map(str, val)) + ')' data_dir_escaped = data_dir_clean.replace('\\', '\\\\')
elif isinstance(val, bool):
return str(val) # Calculate annotations directory (where JSON files are stored)
elif isinstance(val, str): # This is in the output folder, not with the images
return f'"{val}"' from models.TrainingProjectDetails import TrainingProjectDetails
else: details = TrainingProjectDetails.query.get(project_details_id)
return str(val) if details:
training_project = TrainingProject.query.get(details.project_id)
# Add all config parameters to exp project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details.project_id}'
for key, value in config.items(): else:
if key not in ['exp_name']: # exp_name is handled separately project_name = f'project_{project_details_id}'
exp_content += f" self.{key} = {format_value(value)}\n"
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
# Add exp_name at the end (uses dynamic path) training_folder_name = training_folder_name.replace(' ', '_')
exp_content += f''' self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
''' output_base_path = get_setting('yolox_output_path', './backend')
annotations_parent_dir = os.path.join(output_base_path, project_name, training_folder_name)
return exp_content annotations_parent_escaped = annotations_parent_dir.replace('\\', '\\\\')
def save_yolox_inference_exp(training_id, out_path, options=None): # Set output directory for checkpoints - models subdirectory
"""Save inference exp.py to custom path""" models_dir = os.path.join(annotations_parent_dir, 'models')
exp_content = generate_yolox_inference_exp(training_id, options, use_base_config=False) models_dir_escaped = models_dir.replace('\\', '\\\\')
with open(out_path, 'w') as f:
f.write(exp_content) # Build exp content
return out_path exp_content = f'''#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "{data_dir_escaped}" # Where images are located
self.annotations_dir = "{annotations_parent_escaped}" # Where annotation JSONs are located
self.output_dir = "{models_dir_escaped}" # Where checkpoints will be saved
self.train_ann = "{train_ann}"
self.val_ann = "{val_ann}"
self.test_ann = "{test_ann}"
self.num_classes = {num_classes}
# Disable train2017 subdirectory - our images are directly in data_dir
self.name = ""
# Set data workers for training
self.data_num_workers = 8
'''
# Set pretrained_ckpt if transfer_learning is 'coco'
if training.transfer_learning and isinstance(training.transfer_learning, str) and training.transfer_learning.lower() == 'coco':
yolox_base_dir = '/home/kitraining/Yolox/YOLOX-main'
selected_model = training.selected_model.replace('.pth', '') if training.selected_model else ''
if selected_model:
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n"
# Format arrays and values for Python code generation
# Integer-only parameters (sizes, epochs, intervals)
integer_params = {
'input_size', 'test_size', 'random_size', 'max_epoch', 'warmup_epochs',
'no_aug_epochs', 'print_interval', 'eval_interval', 'multiscale_range',
'data_num_workers', 'num_classes'
}
def format_value(val, param_name=''):
if isinstance(val, (list, tuple)):
# Check if this parameter should have integer values
if param_name in integer_params:
# Convert all values to integers
formatted_items = [str(int(float(item))) if isinstance(item, (int, float)) else str(item) for item in val]
else:
# Keep as floats or original type
formatted_items = []
for item in val:
if isinstance(item, float):
formatted_items.append(str(item))
elif isinstance(item, int):
formatted_items.append(str(item))
else:
formatted_items.append(str(item))
return '(' + ', '.join(formatted_items) + ')'
elif isinstance(val, bool):
return str(val)
elif isinstance(val, str):
return f'"{val}"'
elif isinstance(val, int):
return str(val)
elif isinstance(val, float):
return str(val)
else:
return str(val)
# Add all config parameters to exp
for key, value in config.items():
if key not in ['exp_name']: # exp_name is handled separately
exp_content += f" self.{key} = {format_value(value, key)}\n"
# Add get_dataset override using name parameter for image directory
exp_content += '''
def get_dataset(self, cache=False, cache_type="ram"):
"""Override to use name parameter for images directory"""
from yolox.data import COCODataset
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
# Use empty string for name since we have absolute paths in JSON
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.train_ann,
name="",
img_size=self.input_size,
preproc=self.preproc if hasattr(self, 'preproc') else None,
cache=cache,
cache_type=cache_type,
)
def get_eval_dataset(self, **kwargs):
"""Override eval dataset using name parameter"""
from yolox.data import COCODataset, ValTransform
testdev = kwargs.get("testdev", False)
legacy = kwargs.get("legacy", False)
return COCODataset(
data_dir=self.annotations_dir,
json_file=self.val_ann if not testdev else self.test_ann,
name="",
img_size=self.test_size,
preproc=ValTransform(legacy=legacy), # Use proper validation transform
)
def get_eval_loader(self, batch_size, is_distributed, **kwargs):
"""Standard YOLOX eval loader - matches official implementation"""
import torch
import torch.distributed as dist
from torch.utils.data import DataLoader
valdataset = self.get_eval_dataset(**kwargs)
if is_distributed:
batch_size = batch_size // dist.get_world_size()
sampler = torch.utils.data.distributed.DistributedSampler(
valdataset, shuffle=False
)
else:
sampler = torch.utils.data.SequentialSampler(valdataset)
dataloader_kwargs = {
"num_workers": self.data_num_workers,
"pin_memory": True,
"sampler": sampler,
}
dataloader_kwargs["batch_size"] = batch_size
val_loader = DataLoader(valdataset, **dataloader_kwargs)
return val_loader
'''
# Add exp_name at the end (uses dynamic path)
exp_content += f''' self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
'''
return exp_content
def save_yolox_inference_exp(training_id, out_path, options=None):
"""Save inference exp.py to custom path"""
exp_content = generate_yolox_inference_exp(training_id, options, use_base_config=False)
with open(out_path, 'w') as f:
f.write(exp_content)
return out_path

48
backend/services/push-yolox-exp.js
View File

@@ -1,48 +0,0 @@
const Training = require('../models/training.js');
const fs = require('fs');
const path = require('path');
async function pushYoloxExpToDb(settings) {
// Normalize boolean and array fields for DB
const normalized = { ...settings };
// Map 'act' from frontend to 'activation' for DB
if (normalized.act !== undefined) {
normalized.activation = normalized.act;
delete normalized.act;
}
// Convert 'on'/'off' to boolean for save_history_ckpt
if (typeof normalized.save_history_ckpt === 'string') {
normalized.save_history_ckpt = normalized.save_history_ckpt === 'on' ? true : false;
}
// Convert comma-separated strings to arrays for input_size, test_size, mosaic_scale, mixup_scale
['input_size', 'test_size', 'mosaic_scale', 'mixup_scale'].forEach(key => {
if (typeof normalized[key] === 'string') {
const arr = normalized[key].split(',').map(v => parseFloat(v.trim()));
normalized[key] = arr.length === 1 ? arr[0] : arr;
}
});
// Find TrainingProjectDetails for this project
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
const details = await TrainingProjectDetails.findOne({ where: { project_id: normalized.project_id } });
if (!details) throw new Error('TrainingProjectDetails not found for project_id ' + normalized.project_id);
normalized.project_details_id = details.id;
// Create DB row
const training = await Training.create(normalized);
return training;
}
async function generateYoloxExpFromDb(trainingId) {
// Fetch training row from DB
const training = await Training.findByPk(trainingId);
if (!training) throw new Error('Training not found');
// Template for exp.py
const expTemplate = `#!/usr/bin/env python3\n# Copyright (c) Megvii Inc. All rights reserved.\n\nimport os\nimport random\n\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nfrom .base_exp import BaseExp\n\n__all__ = [\"Exp\", \"check_exp_value\"]\n\nclass Exp(BaseExp):\n def __init__(self):\n super().__init__()\n\n # ---------------- model config ---------------- #\n self.num_classes = ${training.num_classes || 80}\n self.depth = ${training.depth || 1.00}\n self.width = ${training.width || 1.00}\n self.act = \"${training.activation || training.act || 'silu'}\"\n\n # ---------------- dataloader config ---------------- #\n self.data_num_workers = ${training.data_num_workers || 4}\n self.input_size = (${Array.isArray(training.input_size) ? training.input_size.join(', ') : '640, 640'})\n self.multiscale_range = ${training.multiscale_range || 5}\n self.data_dir = ${training.data_dir ? `\"${training.data_dir}\"` : 'None'}\n self.train_ann = \"${training.train_ann || 'instances_train2017.json'}\"\n self.val_ann = \"${training.val_ann || 'instances_val2017.json'}\"\n self.test_ann = \"${training.test_ann || 'instances_test2017.json'}\"\n\n # --------------- transform config ----------------- #\n self.mosaic_prob = ${training.mosaic_prob !== undefined ? training.mosaic_prob : 1.0}\n self.mixup_prob = ${training.mixup_prob !== undefined ? training.mixup_prob : 1.0}\n self.hsv_prob = ${training.hsv_prob !== undefined ? training.hsv_prob : 1.0}\n self.flip_prob = ${training.flip_prob !== undefined ? training.flip_prob : 0.5}\n self.degrees = ${training.degrees !== undefined ? training.degrees : 10.0}\n self.translate = ${training.translate !== undefined ? training.translate : 0.1}\n self.mosaic_scale = (${Array.isArray(training.mosaic_scale) ? training.mosaic_scale.join(', ') : '0.1, 2'})\n self.enable_mixup = ${training.enable_mixup !== undefined ? training.enable_mixup : true}\n self.mixup_scale = (${Array.isArray(training.mixup_scale) ? training.mixup_scale.join(', ') : '0.5, 1.5'})\n self.shear = ${training.shear !== undefined ? training.shear : 2.0}\n\n # -------------- training config --------------------- #\n self.warmup_epochs = ${training.warmup_epochs !== undefined ? training.warmup_epochs : 5}\n self.max_epoch = ${training.max_epoch !== undefined ? training.max_epoch : 300}\n self.warmup_lr = ${training.warmup_lr !== undefined ? training.warmup_lr : 0}\n self.min_lr_ratio = ${training.min_lr_ratio !== undefined ? training.min_lr_ratio : 0.05}\n self.basic_lr_per_img = ${training.basic_lr_per_img !== undefined ? training.basic_lr_per_img : 0.01 / 64.0}\n self.scheduler = \"${training.scheduler || 'yoloxwarmcos'}\"\n self.no_aug_epochs = ${training.no_aug_epochs !== undefined ? training.no_aug_epochs : 15}\n self.ema = ${training.ema !== undefined ? training.ema : true}\n self.weight_decay = ${training.weight_decay !== undefined ? training.weight_decay : 5e-4}\n self.momentum = ${training.momentum !== undefined ? training.momentum : 0.9}\n self.print_interval = ${training.print_interval !== undefined ? training.print_interval : 10}\n self.eval_interval = ${training.eval_interval !== undefined ? training.eval_interval : 10}\n self.save_history_ckpt = ${training.save_history_ckpt !== undefined ? training.save_history_ckpt : true}\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n\n # ----------------- testing config ------------------ #\n self.test_size = (${Array.isArray(training.test_size) ? training.test_size.join(', ') : '640, 640'})\n self.test_conf = ${training.test_conf !== undefined ? training.test_conf : 0.01}\n self.nmsthre = ${training.nmsthre !== undefined ? training.nmsthre : 0.65}\n\n # ... rest of the template ...\n\ndef check_exp_value(exp: Exp):\n h, w = exp.input_size\n assert h % 32 == 0 and w % 32 == 0, \"input size must be multiples of 32\"\n`;
// Save to file in output directory
const outDir = path.join(__dirname, '../../', training.project_id ? `project_${training.project_id}/${trainingId}` : 'exp_files');
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
const filePath = path.join(outDir, 'exp.py');
fs.writeFileSync(filePath, expTemplate);
return filePath;
}
module.exports = { pushYoloxExpToDb, generateYoloxExpFromDb };

205
backend/services/push_yolox_exp.py Normal file → Executable file
View File

@@ -1,92 +1,113 @@
from models.training import Training from models.training import Training
from models.TrainingProjectDetails import TrainingProjectDetails from models.TrainingProjectDetails import TrainingProjectDetails
from database.database import db from models.TrainingSize import TrainingSize
from database.database import db
def push_yolox_exp_to_db(settings):
"""Save YOLOX settings to database""" def push_yolox_exp_to_db(settings):
normalized = dict(settings) """Save YOLOX settings to database"""
normalized = dict(settings)
# Map common frontend aliases to DB column names
alias_map = { # Map common frontend aliases to DB column names
'act': 'activation', alias_map = {
'nmsthre': 'nms_thre', 'act': 'activation',
'select_model': 'selected_model' 'nmsthre': 'nms_thre',
} 'select_model': 'selected_model'
for a, b in alias_map.items(): }
if a in normalized and b not in normalized: for a, b in alias_map.items():
normalized[b] = normalized.pop(a) if a in normalized and b not in normalized:
normalized[b] = normalized.pop(a)
# Convert 'on'/'off' or 'true'/'false' strings to boolean for known boolean fields
for bool_field in ['save_history_ckpt', 'ema', 'enable_mixup']: # Convert 'on'/'off' or 'true'/'false' strings to boolean for known boolean fields
if bool_field in normalized: for bool_field in ['save_history_ckpt', 'ema', 'enable_mixup']:
val = normalized[bool_field] if bool_field in normalized:
if isinstance(val, str): val = normalized[bool_field]
normalized[bool_field] = val.lower() in ('1', 'true', 'on') if isinstance(val, str):
else: normalized[bool_field] = val.lower() in ('1', 'true', 'on')
normalized[bool_field] = bool(val) else:
normalized[bool_field] = bool(val)
# Convert comma-separated strings to arrays for JSON fields
for key in ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale']: # Extract size arrays for separate TrainingSize table (3NF)
if key in normalized and isinstance(normalized[key], str): size_arrays = {}
parts = [p.strip() for p in normalized[key].split(',') if p.strip()] for key in ['input_size', 'test_size', 'mosaic_scale', 'mixup_scale']:
try: if key in normalized:
arr = [float(p) for p in parts] if isinstance(normalized[key], str):
except Exception: parts = [p.strip() for p in normalized[key].split(',') if p.strip()]
arr = parts try:
normalized[key] = arr[0] if len(arr) == 1 else arr arr = [float(p) for p in parts]
except Exception:
# Ensure we have a TrainingProjectDetails row for project_id arr = parts
project_id = normalized.get('project_id') size_arrays[key] = arr[0] if len(arr) == 1 else (arr if isinstance(arr, list) else [arr])
if not project_id: elif isinstance(normalized[key], list):
raise Exception('Missing project_id in settings') size_arrays[key] = normalized[key]
details = TrainingProjectDetails.query.filter_by(project_id=project_id).first() elif normalized[key] is not None:
if not details: size_arrays[key] = [float(normalized[key])]
raise Exception(f'TrainingProjectDetails not found for project_id {project_id}') # Remove from normalized dict since it won't be stored in training table
normalized['project_details_id'] = details.id del normalized[key]
# Filter normalized to only columns that exist on the Training model # Ensure we have a TrainingProjectDetails row for project_id
valid_cols = {c.name: c for c in Training.__table__.columns} project_id = normalized.get('project_id')
filtered = {} if not project_id:
for k, v in normalized.items(): raise Exception('Missing project_id in settings')
if k in valid_cols: details = TrainingProjectDetails.query.filter_by(project_id=project_id).first()
col_type = valid_cols[k].type.__class__.__name__ if not details:
# Try to coerce types for numeric/boolean columns raise Exception(f'TrainingProjectDetails not found for project_id {project_id}')
try: normalized['project_details_id'] = details.id
if 'Integer' in col_type:
if v is None or v == '': # Filter normalized to only columns that exist on the Training model
filtered[k] = None valid_cols = {c.name: c for c in Training.__table__.columns}
else: filtered = {}
filtered[k] = int(float(v)) for k, v in normalized.items():
elif 'Float' in col_type: if k in valid_cols:
if v is None or v == '': col_type = valid_cols[k].type.__class__.__name__
filtered[k] = None # Try to coerce types for numeric/boolean columns
else: try:
filtered[k] = float(v) if 'Integer' in col_type:
elif 'Boolean' in col_type: if v is None or v == '':
if isinstance(v, str): filtered[k] = None
filtered[k] = v.lower() in ('1', 'true', 'on') else:
else: filtered[k] = int(float(v))
filtered[k] = bool(v) elif 'Float' in col_type:
elif 'JSON' in col_type: if v is None or v == '':
filtered[k] = v filtered[k] = None
elif 'LargeBinary' in col_type: else:
# If a file path was passed, store its bytes; otherwise store raw bytes filtered[k] = float(v)
if isinstance(v, str): elif 'Boolean' in col_type:
try: if isinstance(v, str):
filtered[k] = v.encode('utf-8') filtered[k] = v.lower() in ('1', 'true', 'on')
except Exception: else:
filtered[k] = None filtered[k] = bool(v)
else: elif 'LargeBinary' in col_type:
filtered[k] = v # If a file path was passed, store its bytes; otherwise store raw bytes
else: if isinstance(v, str):
filtered[k] = v try:
except Exception: filtered[k] = v.encode('utf-8')
# If conversion fails, just assign raw value except Exception:
filtered[k] = v filtered[k] = None
else:
# Create DB row filtered[k] = v
training = Training(**filtered) else:
db.session.add(training) filtered[k] = v
db.session.commit() except Exception:
# If conversion fails, just assign raw value
return training filtered[k] = v
# Create DB row
training = Training(**filtered)
db.session.add(training)
db.session.flush() # Get training.id
# Save size arrays to TrainingSize table (3NF)
for size_type, values in size_arrays.items():
if values and isinstance(values, list):
for order, value in enumerate(values):
size_record = TrainingSize(
training_id=training.id,
size_type=size_type,
value_order=order,
value=float(value)
)
db.session.add(size_record)
db.session.commit()
return training

120
backend/services/seed-label-studio.js
View File

@@ -1,120 +0,0 @@
const sequelize = require('../database/database.js');
const { Project, Img, Ann } = require('../models');
const { fetchLableStudioProject, fetchProjectIdsAndTitles } = require('./fetch-labelstudio.js');
const updateStatus = { running: false };
async function seedLabelStudio() {
updateStatus.running = true;
console.log('Seeding started');
try {
await sequelize.sync();
const projects = await fetchProjectIdsAndTitles();
for (const project of projects) {
console.log(`Processing project ${project.id} (${project.title})`);
// Upsert project in DB
await Project.upsert({ project_id: project.id, title: project.title });
// Fetch project data (annotations array)
const data = await fetchLableStudioProject(project.id);
if (!Array.isArray(data) || data.length === 0) {
console.log(`No annotation data for project ${project.id}`);
continue;
}
// Remove old images and annotations for this project
const oldImages = await Img.findAll({ where: { project_id: project.id } });
const oldImageIds = oldImages.map(img => img.image_id);
if (oldImageIds.length > 0) {
await Ann.destroy({ where: { image_id: oldImageIds } });
await Img.destroy({ where: { project_id: project.id } });
console.log(`Deleted ${oldImageIds.length} old images and their annotations for project ${project.id}`);
}
// Prepare arrays
const imagesBulk = [];
const annsBulk = [];
for (const ann of data) {
// Extract width/height
let width = null;
let height = null;
if (Array.isArray(ann.label_rectangles) && ann.label_rectangles.length > 0) {
width = ann.label_rectangles[0].original_width;
height = ann.label_rectangles[0].original_height;
} else if (Array.isArray(ann.label) && ann.label.length > 0 && ann.label[0].original_width && ann.label[0].original_height) {
width = ann.label[0].original_width;
height = ann.label[0].original_height;
}
// Only push image and annotations if width and height are valid
if (width && height) {
imagesBulk.push({
project_id: project.id,
image_path: ann.image,
width,
height
});
// Handle multiple annotations per image
if (Array.isArray(ann.label_rectangles)) {
for (const ann_detail of ann.label_rectangles) {
annsBulk.push({
image_path: ann.image,
x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
});
}
} else if (Array.isArray(ann.label)) {
for (const ann_detail of ann.label) {
annsBulk.push({
image_path: ann.image,
x: (ann_detail.x * width) / 100,
y: (ann_detail.y * height) / 100,
width: (ann_detail.width * width) / 100,
height: (ann_detail.height * height) / 100,
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
});
}
}
}
}
// 1) Insert images and get generated IDs
const insertedImages = await Img.bulkCreate(imagesBulk, { returning: true });
// 2) Map image_path -> image_id
const imageMap = {};
for (const img of insertedImages) {
imageMap[img.image_path] = img.image_id;
}
// 3) Assign correct image_id to each annotation
for (const ann of annsBulk) {
ann.image_id = imageMap[ann.image_path];
delete ann.image_path; // cleanup
}
// 4) Insert annotations
await Ann.bulkCreate(annsBulk);
console.log(`Inserted ${imagesBulk.length} images and ${annsBulk.length} annotations for project ${project.id}`);
}
console.log('Seeding done');
return { success: true, message: 'Data inserted successfully!' };
} catch (error) {
console.error('Error inserting data:', error);
return { success: false, message: error.message };
} finally {
updateStatus.running = false;
console.log('updateStatus.running set to false');
}
}
module.exports = { seedLabelStudio, updateStatus };

298
backend/services/seed_label_studio.py Normal file → Executable file
View File

@@ -1,149 +1,149 @@
from database.database import db from database.database import db
from models.LabelStudioProject import LabelStudioProject from models.LabelStudioProject import LabelStudioProject
from models.Images import Image from models.Images import Image
from models.Annotation import Annotation from models.Annotation import Annotation
from services.fetch_labelstudio import fetch_label_studio_project, fetch_project_ids_and_titles from services.fetch_labelstudio import fetch_label_studio_project, fetch_project_ids_and_titles
update_status = {"running": False} update_status = {"running": False}
def seed_label_studio(): def seed_label_studio():
"""Seed database with Label Studio project data""" """Seed database with Label Studio project data"""
update_status["running"] = True update_status["running"] = True
print('Seeding started') print('Seeding started')
try: try:
projects = fetch_project_ids_and_titles() projects = fetch_project_ids_and_titles()
for project in projects: for project in projects:
print(f"Processing project {project['id']} ({project['title']})") print(f"Processing project {project['id']} ({project['title']})")
# Upsert project in DB # Upsert project in DB
existing_project = LabelStudioProject.query.filter_by(project_id=project['id']).first() existing_project = LabelStudioProject.query.filter_by(project_id=project['id']).first()
if existing_project: if existing_project:
existing_project.title = project['title'] existing_project.title = project['title']
else: else:
new_project = LabelStudioProject(project_id=project['id'], title=project['title']) new_project = LabelStudioProject(project_id=project['id'], title=project['title'])
db.session.add(new_project) db.session.add(new_project)
db.session.commit() db.session.commit()
# Fetch project data (annotations array) # Fetch project data (annotations array)
data = fetch_label_studio_project(project['id']) data = fetch_label_studio_project(project['id'])
if not isinstance(data, list) or len(data) == 0: if not isinstance(data, list) or len(data) == 0:
print(f"No annotation data for project {project['id']}") print(f"No annotation data for project {project['id']}")
continue continue
# Remove old images and annotations for this project # Remove old images and annotations for this project
old_images = Image.query.filter_by(project_id=project['id']).all() old_images = Image.query.filter_by(project_id=project['id']).all()
old_image_ids = [img.image_id for img in old_images] old_image_ids = [img.image_id for img in old_images]
if old_image_ids: if old_image_ids:
Annotation.query.filter(Annotation.image_id.in_(old_image_ids)).delete(synchronize_session=False) Annotation.query.filter(Annotation.image_id.in_(old_image_ids)).delete(synchronize_session=False)
Image.query.filter_by(project_id=project['id']).delete() Image.query.filter_by(project_id=project['id']).delete()
db.session.commit() db.session.commit()
print(f"Deleted {len(old_image_ids)} old images and their annotations for project {project['id']}") print(f"Deleted {len(old_image_ids)} old images and their annotations for project {project['id']}")
# Prepare arrays # Prepare arrays
images_bulk = [] images_bulk = []
anns_bulk = [] anns_bulk = []
for ann in data: for ann in data:
# Extract width/height # Extract width/height
width = None width = None
height = None height = None
if isinstance(ann.get('label_rectangles'), list) and len(ann['label_rectangles']) > 0: if isinstance(ann.get('label_rectangles'), list) and len(ann['label_rectangles']) > 0:
width = ann['label_rectangles'][0].get('original_width') width = ann['label_rectangles'][0].get('original_width')
height = ann['label_rectangles'][0].get('original_height') height = ann['label_rectangles'][0].get('original_height')
elif isinstance(ann.get('label'), list) and len(ann['label']) > 0: elif isinstance(ann.get('label'), list) and len(ann['label']) > 0:
if ann['label'][0].get('original_width') and ann['label'][0].get('original_height'): if ann['label'][0].get('original_width') and ann['label'][0].get('original_height'):
width = ann['label'][0]['original_width'] width = ann['label'][0]['original_width']
height = ann['label'][0]['original_height'] height = ann['label'][0]['original_height']
# Only process if width and height are valid # Only process if width and height are valid
if width and height: if width and height:
image_data = { image_data = {
'project_id': project['id'], 'project_id': project['id'],
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'width': width, 'width': int(width), # Ensure integer
'height': height 'height': int(height) # Ensure integer
} }
images_bulk.append(image_data) images_bulk.append(image_data)
# Handle multiple annotations per image # Handle multiple annotations per image
if isinstance(ann.get('label_rectangles'), list): if isinstance(ann.get('label_rectangles'), list):
for ann_detail in ann['label_rectangles']: for ann_detail in ann['label_rectangles']:
# Get label safely # Get label safely
rectanglelabels = ann_detail.get('rectanglelabels', []) rectanglelabels = ann_detail.get('rectanglelabels', [])
if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0: if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0:
label = rectanglelabels[0] label = rectanglelabels[0]
elif isinstance(rectanglelabels, str): elif isinstance(rectanglelabels, str):
label = rectanglelabels label = rectanglelabels
else: else:
label = 'unknown' label = 'unknown'
ann_data = { ann_data = {
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'x': (ann_detail['x'] * width) / 100, 'x': (ann_detail['x'] * width) / 100,
'y': (ann_detail['y'] * height) / 100, 'y': (ann_detail['y'] * height) / 100,
'width': (ann_detail['width'] * width) / 100, 'width': (ann_detail['width'] * width) / 100,
'height': (ann_detail['height'] * height) / 100, 'height': (ann_detail['height'] * height) / 100,
'Label': label 'Label': label
} }
anns_bulk.append(ann_data) anns_bulk.append(ann_data)
elif isinstance(ann.get('label'), list): elif isinstance(ann.get('label'), list):
for ann_detail in ann['label']: for ann_detail in ann['label']:
# Get label safely # Get label safely
rectanglelabels = ann_detail.get('rectanglelabels', []) rectanglelabels = ann_detail.get('rectanglelabels', [])
if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0: if isinstance(rectanglelabels, list) and len(rectanglelabels) > 0:
label = rectanglelabels[0] label = rectanglelabels[0]
elif isinstance(rectanglelabels, str): elif isinstance(rectanglelabels, str):
label = rectanglelabels label = rectanglelabels
else: else:
label = 'unknown' label = 'unknown'
ann_data = { ann_data = {
'image_path': ann.get('image'), 'image_path': ann.get('image'),
'x': (ann_detail['x'] * width) / 100, 'x': (ann_detail['x'] * width) / 100,
'y': (ann_detail['y'] * height) / 100, 'y': (ann_detail['y'] * height) / 100,
'width': (ann_detail['width'] * width) / 100, 'width': (ann_detail['width'] * width) / 100,
'height': (ann_detail['height'] * height) / 100, 'height': (ann_detail['height'] * height) / 100,
'Label': label 'Label': label
} }
anns_bulk.append(ann_data) anns_bulk.append(ann_data)
# Insert images and get generated IDs # Insert images and get generated IDs
inserted_images = [] inserted_images = []
for img_data in images_bulk: for img_data in images_bulk:
new_image = Image(**img_data) new_image = Image(**img_data)
db.session.add(new_image) db.session.add(new_image)
db.session.flush() # Flush to get the ID db.session.flush() # Flush to get the ID
inserted_images.append(new_image) inserted_images.append(new_image)
db.session.commit() db.session.commit()
# Map image_path -> image_id # Map image_path -> image_id
image_map = {img.image_path: img.image_id for img in inserted_images} image_map = {img.image_path: img.image_id for img in inserted_images}
# Assign correct image_id to each annotation # Assign correct image_id to each annotation
for ann_data in anns_bulk: for ann_data in anns_bulk:
ann_data['image_id'] = image_map.get(ann_data['image_path']) ann_data['image_id'] = image_map.get(ann_data['image_path'])
del ann_data['image_path'] del ann_data['image_path']
# Insert annotations # Insert annotations
for ann_data in anns_bulk: for ann_data in anns_bulk:
new_annotation = Annotation(**ann_data) new_annotation = Annotation(**ann_data)
db.session.add(new_annotation) db.session.add(new_annotation)
db.session.commit() db.session.commit()
print(f"Inserted {len(images_bulk)} images and {len(anns_bulk)} annotations for project {project['id']}") print(f"Inserted {len(images_bulk)} images and {len(anns_bulk)} annotations for project {project['id']}")
print('Seeding done') print('Seeding done')
return {'success': True, 'message': 'Data inserted successfully!'} return {'success': True, 'message': 'Data inserted successfully!'}
except Exception as error: except Exception as error:
print(f'Error inserting data: {error}') print(f'Error inserting data: {error}')
db.session.rollback() db.session.rollback()
return {'success': False, 'message': str(error)} return {'success': False, 'message': str(error)}
finally: finally:
update_status["running"] = False update_status["running"] = False
print('updateStatus.running set to false') print('updateStatus.running set to false')

71
backend/services/settings_service.py Executable file
View File

@@ -0,0 +1,71 @@
"""
Settings Service - Manages global application settings
"""
from models.Settings import Settings
from database.database import db
def get_setting(key, default=None):
"""Get a setting value by key"""
setting = Settings.query.filter_by(key=key).first()
return setting.value if setting else default
def set_setting(key, value, description=None):
"""Set a setting value"""
setting = Settings.query.filter_by(key=key).first()
if setting:
setting.value = value
if description:
setting.description = description
else:
setting = Settings(key=key, value=value, description=description)
db.session.add(setting)
db.session.commit()
return setting
def get_all_settings():
"""Get all settings as a dictionary"""
settings = Settings.query.all()
return {s.key: s.value for s in settings}
def get_all_settings_detailed():
"""Get all settings with full details"""
settings = Settings.query.all()
return [s.to_dict() for s in settings]
def initialize_default_settings():
"""Initialize default settings if they don't exist"""
defaults = {
'labelstudio_api_url': {
'value': 'http://192.168.1.19:8080/api',
'description': 'Label Studio API URL'
},
'labelstudio_api_token': {
'value': 'c1cef980b7c73004f4ee880a42839313b863869f',
'description': 'Label Studio API Token'
},
'yolox_path': {
'value': 'C:/YOLOX',
'description': 'Path to YOLOX installation directory'
},
'yolox_venv_path': {
'value': '/home/kitraining/Yolox/yolox_venv/bin/activate',
'description': 'Path to YOLOX virtual environment activation script'
},
'yolox_output_path': {
'value': './backend',
'description': 'Output folder for YOLOX experiment files and JSONs'
},
'yolox_data_dir': {
'value': '/home/kitraining/To_Annotate/',
'description': 'Data directory path for YOLOX training (where images are located)'
}
}
for key, data in defaults.items():
existing = Settings.query.filter_by(key=key).first()
if not existing:
setting = Settings(key=key, value=data['value'], description=data['description'])
db.session.add(setting)
db.session.commit()

206
backend/services/training_queue.py Executable file
View File

@@ -0,0 +1,206 @@
"""
Training Queue Manager
Manages a queue of training jobs and tracks their progress
"""
import threading
import queue
import subprocess
import re
import os
from services.settings_service import get_setting
from models.training import Training
class TrainingQueueManager:
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
if self._initialized:
return
self.queue = queue.Queue()
self.current_training = None
self.current_process = None
self.worker_thread = None
self.running = False
self._initialized = True
# Start the worker thread
self.start_worker()
def start_worker(self):
"""Start the background worker thread"""
if self.worker_thread is None or not self.worker_thread.is_alive():
self.running = True
self.worker_thread = threading.Thread(target=self._process_queue, daemon=True)
self.worker_thread.start()
def add_to_queue(self, training_id, command, cwd):
"""Add a training job to the queue"""
job = {
'training_id': training_id,
'command': command,
'cwd': cwd,
'iteration': 0,
'max_epoch': 300 # Will be updated from training record
}
# Get max_epoch from training record
try:
training = Training.query.get(training_id)
if training:
job['max_epoch'] = training.max_epoch or 300
job['name'] = training.exp_name or f'Training {training_id}'
except:
pass
self.queue.put(job)
print(f'Added training {training_id} to queue. Queue size: {self.queue.qsize()}')
def _process_queue(self):
"""Worker thread that processes the queue"""
while self.running:
try:
# Wait for a job (blocking with timeout)
job = self.queue.get(timeout=1)
print(f'Starting training {job["training_id"]} from queue')
self.current_training = job
# Execute the training command
self._run_training(job)
# Mark as done
self.queue.task_done()
self.current_training = None
self.current_process = None
except queue.Empty:
continue
except Exception as e:
print(f'Error processing training job: {e}')
self.current_training = None
self.current_process = None
def _run_training(self, job):
"""Run a training command and monitor its output"""
try:
import platform
is_windows = platform.system() == 'Windows'
# Start process
self.current_process = subprocess.Popen(
job['command'],
shell=True,
cwd=job['cwd'],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
universal_newlines=True,
bufsize=1
)
# Monitor output for progress
for line in iter(self.current_process.stdout.readline, ''):
if line:
print(line.strip())
# Parse epoch and iteration from YOLOX output
# Example: "epoch: 3/300, iter: 90/101"
epoch_match = re.search(r'epoch:\s*(\d+)/(\d+)', line, re.IGNORECASE)
iter_match = re.search(r'iter:\s*(\d+)/(\d+)', line, re.IGNORECASE)
if epoch_match:
current_epoch = int(epoch_match.group(1))
total_epochs = int(epoch_match.group(2))
if self.current_training:
self.current_training['current_epoch'] = current_epoch
self.current_training['max_epoch'] = total_epochs
# Debug log
print(f'[PROGRESS] Parsed epoch: {current_epoch}/{total_epochs}')
if iter_match:
current_iter = int(iter_match.group(1))
total_iters = int(iter_match.group(2))
if self.current_training:
self.current_training['current_iter'] = current_iter
self.current_training['total_iters'] = total_iters
# Calculate overall progress percentage
if 'current_epoch' in self.current_training and 'max_epoch' in self.current_training:
epoch_progress = (self.current_training['current_epoch'] - 1) / self.current_training['max_epoch']
iter_progress = current_iter / total_iters / self.current_training['max_epoch']
total_progress = (epoch_progress + iter_progress) * 100
self.current_training['progress'] = round(total_progress, 2)
# Debug log
print(f'[PROGRESS] Epoch {self.current_training["current_epoch"]}/{self.current_training["max_epoch"]}, Iter {current_iter}/{total_iters}, Progress: {self.current_training["progress"]}%')
# Wait for completion
self.current_process.wait()
print(f'Training {job["training_id"]} completed with exit code {self.current_process.returncode}')
except Exception as e:
print(f'Error running training: {e}')
def get_status(self):
"""Get current status of training queue"""
queue_items = []
# Get items from queue without removing them
temp_items = []
while not self.queue.empty():
try:
item = self.queue.get_nowait()
temp_items.append(item)
queue_items.append({
'training_id': item['training_id'],
'name': item.get('name', f'Training {item["training_id"]}'),
'max_epoch': item.get('max_epoch', 300)
})
except queue.Empty:
break
# Put items back
for item in temp_items:
self.queue.put(item)
result = {
'current': None,
'queue': queue_items
}
if self.current_training:
current_epoch = self.current_training.get('current_epoch', 0)
max_epoch = self.current_training.get('max_epoch', 300)
result['current'] = {
'training_id': self.current_training['training_id'],
'name': self.current_training.get('name', f'Training {self.current_training["training_id"]}'),
'epoch': current_epoch, # For backward compatibility
'current_epoch': current_epoch,
'max_epoch': max_epoch,
'current_iter': self.current_training.get('current_iter', 0),
'total_iters': self.current_training.get('total_iters', 0),
'progress': self.current_training.get('progress', 0.0),
'iteration': current_epoch # For backward compatibility
}
return result
def stop(self):
"""Stop the worker thread"""
self.running = False
if self.current_process:
try:
self.current_process.terminate()
except:
pass
# Global instance
training_queue = TrainingQueueManager()

244
backend/services/validate_dataset.py Executable file
View File

@@ -0,0 +1,244 @@
"""
Validate dataset for training - check for problematic images and annotations
"""
import os
import json
from PIL import Image
import cv2
def validate_coco_json(json_path, data_dir):
"""
Validate a COCO JSON file and check all images
Args:
json_path: Path to COCO JSON file
data_dir: Directory where images are located
Returns:
dict with validation results
"""
print(f"\n{'='*60}")
print(f"Validating: {json_path}")
print(f"{'='*60}\n")
issues = {
'missing_images': [],
'corrupted_images': [],
'zero_dimension_images': [],
'invalid_annotations': [],
'zero_area_boxes': []
}
try:
with open(json_path, 'r') as f:
coco_data = json.load(f)
except Exception as e:
print(f"❌ Failed to load JSON: {e}")
return issues
images = coco_data.get('images', [])
annotations = coco_data.get('annotations', [])
print(f"📊 Dataset Stats:")
print(f" Images: {len(images)}")
print(f" Annotations: {len(annotations)}")
print(f" Categories: {len(coco_data.get('categories', []))}")
print()
# Validate images
print("🔍 Validating images...")
for idx, img_info in enumerate(images):
img_id = img_info.get('id')
file_name = img_info.get('file_name', '')
width = img_info.get('width', 0)
height = img_info.get('height', 0)
# Check if image file exists
# Try to construct the full path
if os.path.isabs(file_name):
img_path = file_name
else:
img_path = os.path.join(data_dir, file_name)
if not os.path.exists(img_path):
issues['missing_images'].append({
'id': img_id,
'file_name': file_name,
'expected_path': img_path
})
continue
# Check if image can be loaded
try:
# Try with PIL
with Image.open(img_path) as pil_img:
pil_width, pil_height = pil_img.size
# Check if dimensions match JSON
if pil_width != width or pil_height != height:
print(f"⚠️ Image {img_id}: Dimension mismatch - JSON: {width}x{height}, Actual: {pil_width}x{pil_height}")
# Check for zero dimensions
if pil_width == 0 or pil_height == 0:
issues['zero_dimension_images'].append({
'id': img_id,
'file_name': file_name,
'dimensions': f"{pil_width}x{pil_height}"
})
except Exception as e:
issues['corrupted_images'].append({
'id': img_id,
'file_name': file_name,
'error': str(e)
})
# Progress indicator
if (idx + 1) % 100 == 0:
print(f" Checked {idx + 1}/{len(images)} images...")
print(f"✅ Image validation complete\n")
# Validate annotations
print("🔍 Validating annotations...")
for idx, ann in enumerate(annotations):
ann_id = ann.get('id')
img_id = ann.get('image_id')
bbox = ann.get('bbox', [])
if len(bbox) != 4:
issues['invalid_annotations'].append({
'id': ann_id,
'image_id': img_id,
'reason': f'Invalid bbox length: {len(bbox)}'
})
continue
x, y, w, h = bbox
# Check for zero or negative dimensions
if w <= 0 or h <= 0:
issues['zero_area_boxes'].append({
'id': ann_id,
'image_id': img_id,
'bbox': bbox,
'reason': f'Zero or negative dimensions: w={w}, h={h}'
})
# Check for extremely small boxes (potential issue with mixup)
if w < 1 or h < 1:
issues['zero_area_boxes'].append({
'id': ann_id,
'image_id': img_id,
'bbox': bbox,
'reason': f'Extremely small box: w={w}, h={h}'
})
# Progress indicator
if (idx + 1) % 1000 == 0:
print(f" Checked {idx + 1}/{len(annotations)} annotations...")
print(f"✅ Annotation validation complete\n")
# Print summary
print(f"\n{'='*60}")
print("VALIDATION SUMMARY")
print(f"{'='*60}\n")
total_issues = sum(len(v) for v in issues.values())
if total_issues == 0:
print("✅ No issues found! Dataset is ready for training.")
else:
print(f"⚠️ Found {total_issues} total issues:\n")
if issues['missing_images']:
print(f" ❌ Missing images: {len(issues['missing_images'])}")
for item in issues['missing_images'][:5]: # Show first 5
print(f" - {item['file_name']}")
if len(issues['missing_images']) > 5:
print(f" ... and {len(issues['missing_images']) - 5} more")
if issues['corrupted_images']:
print(f" ❌ Corrupted images: {len(issues['corrupted_images'])}")
for item in issues['corrupted_images'][:5]:
print(f" - {item['file_name']}: {item['error']}")
if len(issues['corrupted_images']) > 5:
print(f" ... and {len(issues['corrupted_images']) - 5} more")
if issues['zero_dimension_images']:
print(f" ❌ Zero dimension images: {len(issues['zero_dimension_images'])}")
for item in issues['zero_dimension_images'][:5]:
print(f" - {item['file_name']}: {item['dimensions']}")
if len(issues['zero_dimension_images']) > 5:
print(f" ... and {len(issues['zero_dimension_images']) - 5} more")
if issues['invalid_annotations']:
print(f" ❌ Invalid annotations: {len(issues['invalid_annotations'])}")
for item in issues['invalid_annotations'][:5]:
print(f" - Ann ID {item['id']}: {item['reason']}")
if len(issues['invalid_annotations']) > 5:
print(f" ... and {len(issues['invalid_annotations']) - 5} more")
if issues['zero_area_boxes']:
print(f" ⚠️ Zero/tiny area boxes: {len(issues['zero_area_boxes'])}")
print(f" These may cause issues with mixup augmentation!")
for item in issues['zero_area_boxes'][:5]:
print(f" - Ann ID {item['id']}, bbox: {item['bbox']}")
if len(issues['zero_area_boxes']) > 5:
print(f" ... and {len(issues['zero_area_boxes']) - 5} more")
print()
return issues
def validate_training_dataset(training_id):
"""
Validate all COCO JSON files for a training
Args:
training_id: The training ID to validate
"""
from models.training import Training
from models.TrainingProject import TrainingProject
from services.settings_service import get_setting
training = Training.query.get(training_id)
if not training:
print(f"❌ Training {training_id} not found")
return
# Get paths
from models.TrainingProjectDetails import TrainingProjectDetails
details = TrainingProjectDetails.query.get(training.project_details_id)
training_project = TrainingProject.query.get(details.project_id)
project_name = training_project.title.replace(' ', '_') if training_project else f'project_{details.project_id}'
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
training_folder_name = training_folder_name.replace(' ', '_')
output_base_path = get_setting('yolox_output_path', './backend')
data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
# Validate each split
splits = ['train', 'valid', 'test']
all_issues = {}
for split in splits:
json_file = os.path.join(annotations_dir, f'coco_project_{training_id}_{split}.json')
if os.path.exists(json_file):
all_issues[split] = validate_coco_json(json_file, data_dir)
else:
print(f"⚠️ JSON file not found: {json_file}")
return all_issues
if __name__ == '__main__':
import sys
if len(sys.argv) > 1:
training_id = int(sys.argv[1])
validate_training_dataset(training_id)
else:
print("Usage: python validate_dataset.py <training_id>")

28
backend/start.py Normal file → Executable file
View File

@@ -1,14 +1,14 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Start the Flask backend server Start the Flask backend server
""" """
import sys import sys
import os import os
# Add the backend directory to Python path # Add the backend directory to Python path
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from app import app from app import app
if __name__ == '__main__': if __name__ == '__main__':
app.run(host='0.0.0.0', port=3000, debug=True) app.run(host='0.0.0.0', port=3000, debug=True)

47
backend/test/7/exp.py Executable file
View File

@@ -0,0 +1,47 @@
#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Copyright (c) Megvii, Inc. and its affiliates.
import os
from yolox.exp import Exp as MyExp
class Exp(MyExp):
def __init__(self):
super(Exp, self).__init__()
self.data_dir = "/home/kitraining/To_Annotate/"
self.train_ann = "coco_project_3_train.json"
self.val_ann = "coco_project_3_valid.json"
self.test_ann = "coco_project_3_test.json"
self.num_classes = 2
self.depth = 1.0
self.width = 1.0
self.input_size = (640.0, 640.0)
self.mosaic_scale = (0.1, 2.0)
self.test_size = (640.0, 640.0)
self.enable_mixup = True
self.max_epoch = 300
self.warmup_epochs = 5
self.warmup_lr = 0.0
self.scheduler = "yoloxwarmcos"
self.no_aug_epochs = 15
self.min_lr_ratio = 0.05
self.ema = True
self.weight_decay = 0.0005
self.momentum = 0.9
self.print_interval = 10
self.eval_interval = 10
self.test_conf = 0.01
self.nms_thre = 0.65
self.mosaic_prob = 1.0
self.mixup_prob = 1.0
self.hsv_prob = 1.0
self.flip_prob = 0.5
self.degrees = 10.0
self.translate = 0.1
self.shear = 2.0
self.mixup_scale = (0.5, 1.5)
self.activation = "silu"
self.random_size = (10, 20)
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]

BIN
documentation/Projektdoku.pdf Executable file
View File

Binary file not shown.

764
documentation/Projektdokumentation.md Executable file
View File

@@ -0,0 +1,764 @@
# Projektdokumentation: mb ai Trainer
**Autor:** [Ihr Name]
**Datum:** 1. Dezember 2025
**Projekt:** mb ai Trainer - Webbasiertes YOLOX Trainings-Management-System
---
## Inhaltsverzeichnis
1. [Einleitung](#1-einleitung)
- 1.1 [Das Unternehmen](#11-das-unternehmen)
- 1.2 [Projektumfeld](#12-projektumfeld)
- 1.3 [Projektziel](#13-projektziel)
- 1.4 [Projektbegründung](#14-projektbegründung)
2. [Grundlagen](#2-grundlagen)
- 2.1 [Verwendete Technologien](#21-verwendete-technologien)
- 2.2 [YOLOX Object Detection](#22-yolox-object-detection)
- 2.3 [Label Studio](#23-label-studio)
3. [Ist-Analyse](#3-ist-analyse)
- 3.1 [Ausgangssituation](#31-ausgangssituation)
- 3.2 [Problembeschreibung](#32-problembeschreibung)
4. [Soll-Konzept](#4-soll-konzept)
- 4.1 [Anforderungsanalyse](#41-anforderungsanalyse)
- 4.2 [Funktionale Anforderungen](#42-funktionale-anforderungen)
- 4.3 [Nicht-funktionale Anforderungen](#43-nicht-funktionale-anforderungen)
5. [Systemarchitektur](#5-systemarchitektur)
- 5.1 [Gesamtarchitektur](#51-gesamtarchitektur)
- 5.2 [Backend-Architektur](#52-backend-architektur)
- 5.3 [Frontend-Architektur](#53-frontend-architektur)
- 5.4 [Datenbankdesign](#54-datenbankdesign)
- 5.5 [API-Design](#55-api-design)
6. [Implementierung](#6-implementierung)
- 6.1 [Backend-Entwicklung](#61-backend-entwicklung)
- 6.2 [Frontend-Entwicklung](#62-frontend-entwicklung)
- 6.3 [Datenbank-Implementierung](#63-datenbank-implementierung)
- 6.4 [Label Studio Integration](#64-label-studio-integration)
- 6.5 [Transfer Learning System](#65-transfer-learning-system)
- 6.6 [Training-Workflow](#66-training-workflow)
7. [Testing und Qualitätssicherung](#7-testing-und-qualitätssicherung)
- 7.1 [Teststrategie](#71-teststrategie)
- 7.2 [Unit Tests](#72-unit-tests)
- 7.3 [Integrationstests](#73-integrationstests)
- 7.4 [Systemtests](#74-systemtests)
8. [Deployment und Betrieb](#8-deployment-und-betrieb)
- 8.1 [Systemanforderungen](#81-systemanforderungen)
- 8.2 [Installation](#82-installation)
- 8.3 [Konfiguration](#83-konfiguration)
9. [Fazit und Ausblick](#9-fazit-und-ausblick)
- 9.1 [Zusammenfassung](#91-zusammenfassung)
- 9.2 [Erreichte Ziele](#92-erreichte-ziele)
- 9.3 [Lessons Learned](#93-lessons-learned)
- 9.4 [Ausblick](#94-ausblick)
10. [Anhang](#10-anhang)
- 10.1 [Glossar](#101-glossar)
- 10.2 [Literaturverzeichnis](#102-literaturverzeichnis)
- 10.3 [Abbildungsverzeichnis](#103-abbildungsverzeichnis)
- 10.4 [Code-Beispiele](#104-code-beispiele)
---
## 1. Einleitung
### 1.1 Das Unternehmen
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Firmenbeschreibung, Standort, Mitarbeiter, Branche, etc.]
### 1.2 Projektumfeld
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Übergeordnetes Projekt, Teamstruktur, Projektorganisation, etc.]
### 1.3 Projektziel
Der **mb ai Trainer** ist eine webbasierte Anwendung zur vollständigen Verwaltung und Durchführung von YOLOX-Modelltrainings im Bereich Computer Vision. Das System wurde entwickelt, um den gesamten Machine Learning Workflow - von der Datenannotation über das Training bis zur Modell-Evaluierung - in einer einheitlichen Plattform zu integrieren.
**Hauptziele des Projekts:**
- Entwicklung einer benutzerfreundlichen Weboberfläche zur Verwaltung von Trainingsprojekten
- Integration von Label Studio für professionelle Bildannotation
- Implementierung einer robusten Datenverwaltung mit MySQL
- Automatisierung des YOLOX-Trainingsprozesses
- Bereitstellung eines Transfer Learning Systems mit vordefinierten Basis-Konfigurationen
- Zentrale Verwaltung aller Trainingsparameter und -verläufe
### 1.4 Projektbegründung
Im Bereich des Machine Learning für Computer Vision existieren zahlreiche isolierte Tools für verschiedene Aufgaben (Annotation, Training, Evaluierung). Dies führt zu:
- **Hohem Verwaltungsaufwand:** Daten müssen zwischen verschiedenen Tools transferiert werden
- **Fehlerpotential:** Manuelle Konfiguration von Trainingsskripten ist fehleranfällig
- **Keine Nachvollziehbarkeit:** Trainingsverläufe und verwendete Parameter sind nicht zentral dokumentiert
- **Steile Lernkurve:** Neue Teammitglieder müssen mehrere Tools erlernen
Der mb ai Trainer löst diese Probleme durch eine integrierte Plattform, die alle notwendigen Funktionen vereint.
---
## 2. Grundlagen
### 2.1 Verwendete Technologien
[HIER INHALT VON KOLLEGE ÜBERNEHMEN - Grundlegende Technologien, allgemeiner Kontext]
#### 2.1.1 Backend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **Python** | 3.x | Hauptsprache für Backend-Entwicklung |
| **Flask** | 3.0.0 | Lightweight Web-Framework für REST API |
| **Flask-CORS** | Latest | Cross-Origin Resource Sharing Support |
| **Flask-SQLAlchemy** | Latest | ORM-Integration in Flask |
| **SQLAlchemy** | Latest | Object-Relational Mapper für Datenbankzugriff |
| **PyMySQL** | Latest | MySQL Database Connector |
| **MySQL** | 8.0+ | Relationale Datenbank für persistente Speicherung |
| **requests** | Latest | HTTP-Client für externe API-Calls (Label Studio) |
**Begründung der Technologiewahl - Backend:**
- **Python:** Dominante Sprache im ML-Bereich, umfangreiches Ökosystem
- **Flask:** Lightweight, flexibel, kein unnötiger Overhead
- **SQLAlchemy:** Mächtiges ORM mit ausgezeichneter MySQL-Unterstützung
- **MySQL:** Bewährtes RDBMS, gut für strukturierte Trainingsdaten
#### 2.1.2 Frontend-Technologien
| Technologie | Version | Verwendung |
|-------------|---------|------------|
| **HTML5** | - | Strukturierung der Weboberfläche |
| **CSS3** | - | Styling, Layout, Responsive Design |
| **Vanilla JavaScript** | ES6+ | Client-seitige Logik ohne Frameworks |
**Begründung der Technologiewahl - Frontend:**
- **Vanilla JavaScript:** Keine Framework-Abhängigkeiten, volle Kontrolle, geringe Dateigröße
- **Kein Build-System:** Einfache Entwicklung und Deployment
- **Moderne ES6+ Features:** Async/Await, Fetch API, Arrow Functions
#### 2.1.3 Externe Services und Tools
| Service/Tool | Verwendung |
|--------------|------------|
| **Label Studio** | Open-Source Annotationstool für Bilddaten |
| **YOLOX** | State-of-the-art Object Detection Framework |
### 2.2 YOLOX Object Detection
YOLOX ist ein modernes, hochperformantes Object Detection Framework, das auf der YOLO-Familie aufbaut.
**Kernmerkmale:**
- **Anchor-free:** Keine vordefinierten Anchor-Boxen notwendig
- **Decoupled Head:** Getrennte Heads für Klassifikation und Lokalisierung
- **SimOTA:** Optimiertes Label Assignment
- **Verschiedene Modellgrößen:** YOLOX-S, -M, -L, -X für unterschiedliche Anforderungen
**Modell-Varianten:**
| Modell | Depth | Width | Parameter | Use Case |
|--------|-------|-------|-----------|----------|
| YOLOX-S | 0.33 | 0.50 | ~9M | Schnelle Inferenz, Edge Devices |
| YOLOX-M | 0.67 | 0.75 | ~25M | Balance Performance/Speed |
| YOLOX-L | 1.00 | 1.00 | ~54M | Hohe Genauigkeit |
| YOLOX-X | 1.33 | 1.25 | ~99M | Maximale Genauigkeit |
### 2.3 Label Studio
Label Studio ist eine Open-Source Daten-Annotationsplattform, die verschiedene Datentypen unterstützt.
**Funktionen:**
- **Multi-Format Support:** Bilder, Text, Audio, Video
- **Flexible Annotation:** Bounding Boxes, Polygone, Segmentierung
- **REST API:** Programmatischer Zugriff auf Projekte und Annotationen
- **Collaboration:** Multi-User Support für Team-Annotationen
**Integration im mb ai Trainer:**
- Import von Bildern aus Label Studio Projekten
- Synchronisation von Annotationen in MySQL-Datenbank
- Export von Annotationen im COCO JSON Format für YOLOX-Training
---
## 3. Ist-Analyse
### 3.1 Ausgangssituation
Vor der Entwicklung des mb ai Trainer existierte ein System basierend auf Node.js, Express und Sequelize ORM. Dieses System wies folgende Charakteristika auf:
**Technischer Stack (Alt):**
- Backend: Node.js mit Express.js
- ORM: Sequelize
- Datenbank: MySQL
- Frontend: HTML/CSS/JavaScript (static)
**Funktionsumfang (Alt):**
- Grundlegende Projektverwaltung
- Einfache Label Studio Anbindung
- Manuelle YOLOX-Konfiguration
### 3.2 Problembeschreibung
Das bestehende System hatte mehrere Limitierungen:
**1. Technologische Inkonsistenz**
- Backend in JavaScript, ML-Framework in Python
- Komplexe Interprozesskommunikation notwendig
- Doppelte Abhängigkeiten (Node.js + Python)
**2. Fehlende Funktionalität**
- Kein Transfer Learning Support
- Keine Basis-Konfigurationen für YOLOX-Modelle
- Manuelle Parameterkonfiguration fehleranfällig
- Keine Validierung von Trainingsparametern
**3. Wartbarkeit**
- Zwei getrennte Technologie-Stacks erschweren Wartung
- Keine einheitliche Codebasis
- Hoher Einarbeitungsaufwand für neue Entwickler
**4. Benutzerfreundlichkeit**
- Keine visuelle Unterscheidung zwischen Pflicht- und optionalen Parametern
- Keine Hilfestellungen für YOLOX-Anfänger
- Kein Schutz vor Fehlkonfigurationen
**Entscheidung:** Komplette Migration des Backends zu Python mit Erweiterung um Transfer Learning Funktionalität.
---
## 4. Soll-Konzept
### 4.1 Anforderungsanalyse
Aus der Ist-Analyse wurden folgende Anforderungen abgeleitet:
### 4.2 Funktionale Anforderungen
#### FA-1: Projektverwaltung
- Anlegen, Bearbeiten, Löschen von Trainingsprojekten
- Upload von Projektbildern
- Verknüpfung mit Label Studio Projekten
#### FA-2: Label Studio Integration
- Abruf von Label Studio Projekten via REST API
- Import von Annotationen
- Synchronisation von Bildern und Annotations in MySQL
- Export im COCO JSON Format
#### FA-3: Training-Konfiguration
- Auswahl von YOLOX-Modellen (S, M, L, X)
- Konfiguration aller Trainingsparameter
- Drei Modi: "Train from scratch", "Train on coco", "Custom weights"
- Validierung von Eingabeparametern
#### FA-4: Transfer Learning System
- Vordefinierte Basis-Konfigurationen für jedes YOLOX-Modell
- Automatisches Laden bei "Train on coco" Modus
- Schutz von Basis-Parametern vor Überschreibung
- Visuelle Kennzeichnung geschützter Felder
#### FA-5: Training-Durchführung
- Generierung von YOLOX exp.py Dateien
- Start des Trainingsprozesses
- Logging von Trainingsverläufen
#### FA-6: Datenbank-Management
- Persistente Speicherung aller Projekte
- Speicherung von Trainingseinstellungen
- Versionierung von Konfigurationen
### 4.3 Nicht-funktionale Anforderungen
#### NFA-1: Performance
- Laden von Basis-Konfigurationen < 100ms
- Label Studio Sync < 5s für 1000 Bilder
- API Response Times < 200ms (95th Percentile)
#### NFA-2: Usability
- Intuitive Benutzeroberfläche ohne Schulung nutzbar
- Klare visuelle Unterscheidung (enabled/disabled Felder)
- Hilfreiche Tooltips und Fehlermeldungen
#### NFA-3: Wartbarkeit
- Modularer Aufbau (Services, Models, Routes getrennt)
- Umfassende Code-Dokumentation
- Einheitlicher Technologie-Stack (Python)
#### NFA-4: Erweiterbarkeit
- Neue YOLOX-Modelle ohne Code-Änderung hinzufügbar
- Basis-Konfigurationen als separate Dateien
- Plugin-Architektur für zukünftige Frameworks
#### NFA-5: Zuverlässigkeit
- Fehlerbehandlung bei API-Calls
- Transaktionale Datenbankoperationen
- Rollback bei fehlgeschlagenen Operationen
---
## 5. Systemarchitektur
### 3.1 Anforderungsanalyse
#### 3.1.1 Funktionale Anforderungen
Das Transfer Learning Feature sollte folgende Funktionen bereitstellen:
1. **Basis-Konfiguration Verwaltung**
- Vordefinierte Konfigurationen für YOLOX-S, -M, -L, -X Modelle
- Speicherung als Python-Klassen für einfache Wartbarkeit
- Dynamisches Laden zur Laufzeit
2. **Parameter-Schutz**
- Bestimmte Parameter (depth, width, etc.) aus Basis-Konfiguration
- Benutzer kann geschützte Parameter nicht überschreiben
- Visuelle Kennzeichnung im Frontend
3. **Merge-Strategie**
- Basis-Konfiguration als Grundlage
- Benutzer-definierte Werte für nicht-geschützte Parameter
- Fallback auf Default-Werte
#### 3.1.2 Nicht-funktionale Anforderungen
- **Performance:** Basis-Konfiguration muss < 100ms laden
- **Usability:** Klare visuelle Unterscheidung zwischen geschützten und editierbaren Feldern
- **Wartbarkeit:** Neue Modelle sollen einfach hinzugefügt werden können
- **Erweiterbarkeit:** System muss für zukünftige YOLOX-Versionen erweiterbar sein
### 3.2 Systemarchitektur
#### 3.2.1 Backend-Architektur
Das Backend des mb ai Trainer ist als modulare Python/Flask-Anwendung aufgebaut:
```
backend/
├── app.py # Hauptanwendung, Flask-Server
├── requirements.txt # Python-Abhängigkeiten
├── database/
│ ├── database.py # SQLAlchemy Konfiguration
│ └── myapp.sql # Datenbankschema
├── models/ # SQLAlchemy Models
│ ├── TrainingProject.py # Trainingsprojekte
│ ├── TrainingProjectDetails.py
│ ├── training.py # Training-Settings
│ ├── LabelStudioProject.py
│ ├── Images.py
│ └── Annotation.js
├── data/ # Basis-Konfigurationen (NEU)
│ ├── yolox_s.py # YOLOX-Small Base Config
│ ├── yolox_m.py # YOLOX-Medium Base Config
│ ├── yolox_l.py # YOLOX-Large Base Config
│ ├── yolox_x.py # YOLOX-XLarge Base Config
│ └── README.md # Dokumentation
├── services/ # Business Logic
│ ├── generate_yolox_exp.py # Exp-Generierung mit Base Config
│ ├── fetch_labelstudio.py # Label Studio API Client
│ ├── seed_label_studio.py # Daten-Seeding
│ ├── generate_json_yolox.py # COCO JSON Export
│ └── push_yolox_exp.py # Settings speichern
└── routes/
└── api.py # REST API Endpoints
```
**Datenfluss:**
1. **Projektanlage:** Frontend → POST `/api/training-projects` → MySQL
2. **Label Studio Sync:** Backend → Label Studio API → MySQL (Images, Annotations)
3. **Transfer Learning:**
- Frontend wählt "Train on coco" + Model
- GET `/api/base-config/<model_name>` → Basis-Config laden
- Frontend sperrt geschützte Felder
4. **Training Start:**
- POST `/api/yolox-settings` → Settings speichern
- POST `/api/start-yolox-training` → Exp.py generieren
- YOLOX Training starten
#### 3.2.2 Basis-Konfiguration Format
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters (protected)
depth = 0.33
width = 0.50
# Training parameters (protected)
scheduler = "yoloxwarmcos"
activation = "silu"
# ... weitere Parameter
```
#### 3.2.3 API Design
**Endpoint:** `GET /api/base-config/<model_name>`
**Response:**
```json
{
"depth": 0.33,
"width": 0.50,
"activation": "silu",
"scheduler": "yoloxwarmcos",
// ... weitere Parameter
}
```
### 3.3 Implementierung
#### 3.3.1 Backend: Dynamic Module Loading
Die Basis-Konfigurationen werden zur Laufzeit dynamisch geladen:
```python
import importlib.util
import os
def load_base_config(selected_model):
"""Dynamically load base configuration for selected model"""
model_map = {
'yolox-s': 'yolox_s.py',
'yolox-m': 'yolox_m.py',
'yolox-l': 'yolox_l.py',
'yolox-x': 'yolox_x.py'
}
file_name = model_map.get(selected_model.lower())
if not file_name:
return None
base_dir = os.path.join(os.path.dirname(__file__), '..', 'data')
file_path = os.path.join(base_dir, file_name)
# Dynamic import
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module.BaseExp
```
**Vorteile:**
- Keine hardcodierten Imports
- Neue Modelle können ohne Code-Änderung hinzugefügt werden
- Klare Trennung von Konfiguration und Logik
#### 3.3.2 Backend: Configuration Merging
Die Merge-Strategie kombiniert drei Quellen:
```python
def generate_yolox_inference_exp(training_id, options, use_base_config=True):
"""Generate exp.py with base config support"""
# 1. Load base configuration
base_config = load_base_config(selected_model) if use_base_config else None
# 2. Load user-defined values from database
training = db.session.get(Training, training_id)
# 3. Merge with priority: base > user > defaults
final_config = {}
# Protected parameters from base config
protected_params = ['depth', 'width', 'activation', 'scheduler', ...]
for param in protected_params:
if base_config and hasattr(base_config, param):
final_config[param] = getattr(base_config, param)
elif hasattr(training, param):
final_config[param] = getattr(training, param)
else:
final_config[param] = DEFAULT_VALUES.get(param)
return final_config
```
**Priorität:**
1. **Base Config** (höchste Priorität bei COCO-Mode)
2. **User Values** (für nicht-geschützte Parameter)
3. **Default Values** (Fallback)
#### 3.3.3 Frontend: Field Locking
Das Frontend zeigt geschützte Felder visuell an:
```javascript
const protectedFields = [
'depth', 'width', 'activation', 'scheduler', 'nmsthre',
'momentum', 'weight_decay', 'warmup_epochs', 'max_epoch',
// ... weitere Felder
];
function applyBaseConfig(config, isCocoMode) {
protectedFields.forEach(field => {
const input = document.querySelector(`[name="${field}"]`);
if (input && config.hasOwnProperty(field)) {
// Set value
input.value = config[field];
if (isCocoMode) {
// Lock field
input.disabled = true;
input.style.backgroundColor = '#d3d3d3';
input.style.cursor = 'not-allowed';
input.title = 'This parameter is locked by base configuration';
}
}
});
// Show/hide info banner
const banner = document.getElementById('base-config-info');
banner.style.display = isCocoMode ? 'block' : 'none';
}
```
**UI-Features:**
- Grauer Hintergrund für gesperrte Felder
- Cursor: `not-allowed` für visuelle Rückmeldung
- Tooltip erklärt warum Feld gesperrt ist
- Grüner Info-Banner zeigt aktive Basis-Konfiguration
#### 3.3.4 Frontend: Field Name Mapping
Alias-Handling für Backend/Frontend Unterschiede:
```javascript
const fieldNameMap = {
'activation': 'act', // Backend: activation, Frontend: act
'nms_thre': 'nmsthre' // Backend: nms_thre, Frontend: nmsthre
};
function applyBaseConfig(config, isCocoMode) {
for (const [backendName, value] of Object.entries(config)) {
// Try direct match first
let input = document.querySelector(`[name="${backendName}"]`);
// Try mapped name if direct match fails
if (!input && fieldNameMap[backendName]) {
const frontendName = fieldNameMap[backendName];
input = document.querySelector(`[name="${frontendName}"]`);
}
if (input) {
input.value = value;
// ... weitere Logik
}
}
}
```
### 3.4 Testing und Validierung
#### 3.4.1 Unit Tests
Erstellte Test-Suite für Basis-Konfigurationen:
```python
# backend/data/test_base_configs.py
import importlib.util
import os
def test_base_config(file_name, expected_depth, expected_width):
"""Test if base config loads correctly"""
file_path = os.path.join(os.path.dirname(__file__), file_name)
spec = importlib.util.spec_from_file_location("base_exp", file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
config = module.BaseExp
assert config.depth == expected_depth
assert config.width == expected_width
# ... weitere Assertions
# Run tests
test_base_config('yolox_s.py', 0.33, 0.50)
test_base_config('yolox_m.py', 0.67, 0.75)
# ...
```
**Ergebnis:** Alle 4 Modell-Konfigurationen laden erfolgreich ✓
#### 3.4.2 Integration Tests
**Testfall 1: COCO Transfer Learning Flow**
1. Dropdown "Train on coco" auswählen → ✓
2. Model "yolox-s" auswählen → ✓
3. Basis-Konfiguration lädt via API → ✓
4. Felder werden ausgefüllt und gesperrt → ✓
5. Info-Banner erscheint → ✓
6. Formular abschicken → ✓
7. Exp.py wird mit gemergten Werten generiert → ✓
**Testfall 2: Sketch Mode**
1. Dropdown "Train from sketch" auswählen → ✓
2. Model-Auswahl wird versteckt → ✓
3. Alle Felder editierbar → ✓
4. Keine Basis-Konfiguration geladen → ✓
**Testfall 3: Field Name Alias**
1. Basis-Config enthält `activation: "silu"` → ✓
2. Frontend-Feld `act` wird korrekt befüllt → ✓
3. Feld wird korrekt gesperrt → ✓
#### 3.4.3 Performance Messungen
| Operation | Durchschnitt | Max |
|-----------|--------------|-----|
| Base Config laden (API) | 45ms | 78ms |
| Frontend Field Update | 12ms | 23ms |
| Exp.py Generierung | 156ms | 234ms |
**Fazit:** Alle Performance-Anforderungen erfüllt ✓
---
## 4. Fazit
### 4.1 Erreichte Ziele
Das Transfer Learning Feature für den mb ai Trainer wurde erfolgreich implementiert und erfüllt alle Anforderungen:
✓ **Basis-Konfiguration System** - 4 YOLOX-Modelle (S, M, L, X) vordefiniert
✓ **Parameter-Schutz** - 24 geschützte Parameter implementiert
✓ **Frontend Integration** - Visuelle Rückmeldung, Field Locking mit HTML/CSS/JS
✓ **Python Backend** - Flask REST API mit SQLAlchemy ORM
✓ **MySQL Integration** - Vollständige Datenbankanbindung
✓ **Label Studio Integration** - Nahtlose Anbindung für Annotationen
✓ **Performance** - Alle Ladezeiten < 100ms
✓ **Dokumentation** - Umfassende Dokumentation erstellt
### 4.2 Herausforderungen
**Problem 1: Backend-Migration von Node.js zu Python**
- Ursprüngliches System war in Node.js/Express/Sequelize entwickelt
- Anforderung: Komplette Migration zu Python/Flask/SQLAlchemy
- Lösung: Vollständige Backend-Konvertierung mit allen Models, Services und Routes
**Problem 2: Label Studio API Integration**
- Synchrone Kommunikation mit externem Service
- Lösung: `requests` Library mit Fehlerbehandlung und Retry-Logik
**Problem 3: Field Name Aliasing**
- Backend nutzt `activation`, Frontend nutzt `act`
- Backend nutzt `nms_thre`, Frontend nutzt `nmsthre`
- Lösung: Mapping-Dictionary für bidirektionales Alias-Handling
**Problem 4: Form Submission mit Disabled Fields**
- HTML disabled Felder werden nicht in FormData inkludiert
- Training würde ohne geschützte Parameter starten
- Lösung: Temporäres Enable vor Submit, dann Re-disable nach Submit
**Problem 5: MySQL Schema-Erweiterung**
- Fehlende Spalten `width` und `height` in `image` Tabelle
- Lösung: ALTER TABLE Statement zur Laufzeit
### 4.3 Mögliche Erweiterungen
1. **Weitere Modelle:** YOLOv8, YOLOv9, EfficientDet Basis-Configs
2. **Custom Base Configs:** Benutzer können eigene Basis-Konfigurationen hochladen
3. **Versionierung:** Historie von Basis-Config Änderungen mit Git-Integration
4. **A/B Testing:** Vergleich verschiedener Basis-Konfigurationen
5. **Cloud-Integration:** Speicherung von Modellen in Cloud Storage
6. **Multi-Tenancy:** Mehrere Benutzer mit separaten Projekten
### 4.4 Persönliches Fazit
Die Implementierung des Transfer Learning Features für den mb ai Trainer war ein erfolgreiches Projekt, das meine Kenntnisse in mehreren Bereichen vertieft hat:
**Technische Kompetenzen:**
- **Python Backend-Entwicklung:** Flask, SQLAlchemy, ORM-Design
- **REST API Design:** Endpoint-Planung, Request/Response-Handling
- **Frontend ohne Framework:** Vanilla JavaScript, DOM-Manipulation, Event-Handling
- **Datenbank-Design:** MySQL Schema-Design, Migrations
- **Externe API-Integration:** Label Studio REST API
- **Merge-Strategien:** Konfigurationsmanagement, Prioritäts-Logik
**Methodische Kompetenzen:**
- Anforderungsanalyse und Systemdesign
- Modulare Architektur und Code-Organisation
- Testing und Validierung
- Technische Dokumentation
Besonders wertvoll war die Erfahrung mit dynamischem Python Module Loading und der Entwicklung einer benutzerfreundlichen UI ausschließlich mit Vanilla JavaScript ohne externe Frameworks.
---
## 5. Anhang
### 5.1 Basis-Konfiguration Beispiel (YOLOX-S)
```python
class BaseExp:
"""Base configuration for YOLOX-S with COCO pretrained weights"""
# Model architecture parameters
depth = 0.33
width = 0.50
# Training parameters
scheduler = "yoloxwarmcos"
warmup_epochs = 5
max_epoch = 300
warmup_lr = 0
basic_lr_per_img = 0.01 / 64.0
# Optimizer parameters
momentum = 0.9
weight_decay = 5e-4
# Augmentation parameters
hsv_prob = 1.0
flip_prob = 0.5
degrees = 10.0
translate = 0.1
mosaic_prob = 1.0
# NMS parameters
nms_thre = 0.65
# Activation
activation = "silu"
```
### 5.2 API Dokumentation
**GET /api/base-config/<model_name>**
Lädt Basis-Konfiguration für angegebenes Modell.
**Parameter:**
- `model_name` (string): yolox-s, yolox-m, yolox-l, oder yolox-x
**Response:** JSON-Objekt mit Konfigurationsparametern
**Status Codes:**
- 200: Erfolg
- 404: Modell nicht gefunden
- 500: Server-Fehler
### 5.3 Verwendete Ressourcen
- Flask Documentation: https://flask.palletsprojects.com/
- Python importlib: https://docs.python.org/3/library/importlib.html
- YOLOX Repository: https://github.com/Megvii-BaseDetection/YOLOX
- MDN Web Docs (FormData): https://developer.mozilla.org/en-US/docs/Web/API/FormData
### 5.4 Glossar
| Begriff | Bedeutung |
|---------|-----------|
| Transfer Learning | Verwendung vortrainierter Modelle als Ausgangspunkt |
| Base Config | Vordefinierte Basis-Konfiguration für Modell |
| Protected Parameters | Parameter die nicht vom Benutzer geändert werden können |
| COCO | Common Objects in Context Dataset |
| YOLOX | State-of-the-art Object Detection Modell |
| Merge Strategy | Logik zum Kombinieren mehrerer Konfigurationsquellen |
---
**Ende der Dokumentation**

1411
edit-training.html Normal file → Executable file
View File

File diff suppressed because it is too large Load Diff

40
globals.css Normal file → Executable file
View File

@@ -1,20 +1,20 @@
@import url("https://cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.min.css"); @import url("https://cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.min.css");
* { * {
-webkit-font-smoothing: antialiased; -webkit-font-smoothing: antialiased;
box-sizing: border-box; box-sizing: border-box;
} }
html, html,
body { body {
height: 100%; height: 100%;
background-color: #f9fafb; background-color: #f9fafb;
} }
/* a blue color as a generic focus style */ /* a blue color as a generic focus style */
button:focus-visible { button:focus-visible {
outline: 2px solid #4a90e2 !important; outline: 2px solid #4a90e2 !important;
outline: -webkit-focus-ring-color auto 5px !important; outline: -webkit-focus-ring-color auto 5px !important;
} }
a { a {
text-decoration: none; text-decoration: none;
} }

393
index.html Normal file → Executable file
View File

@@ -1,96 +1,299 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <!-- Training Notification Bell -->
Training Project</button> <button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
<button id="seed-db-btn" class="button"> style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
Seed Database 🔔
<div class="loader" id="loader" style="display: none"></div> <span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
</button> text-align: center; font-weight: bold;">0</span>
</div> </button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
</div> Training Project</button>
<div id="projects-list"> <button id="seed-db-btn" class="button">
<script src="js/dashboard.js"></script> Seed Database
</div> <div class="loader" id="loader" style="display: none"></div>
<script>
document.getElementById('seed-db-btn').addEventListener('click', function () { </button>
const elLoader = document.getElementById("loader") <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">
elLoader.style.display = "inherit" ⚙️
</button>
fetch('/api/seed') </div>
.finally(() => {
// Instead of hiding loader immediately, poll /api/update-status until done </div>
function pollStatus() { <div id="projects-list">
fetch('/api/update-status') <script src="js/dashboard.js"></script>
.then(res => res.json()) </div>
.then(status => { <script>
if (status && status.running) { document.getElementById('seed-db-btn').addEventListener('click', function () {
// Still running, poll again after short delay const elLoader = document.getElementById("loader")
setTimeout(pollStatus, 5000); elLoader.style.display = "inherit"
} else {
elLoader.style.display = "none"; fetch('/api/seed')
} .finally(() => {
}) // Instead of hiding loader immediately, poll /api/update-status until done
.catch(() => { function pollStatus() {
elLoader.style.display = "none"; fetch('/api/update-status')
}); .then(res => res.json())
} .then(status => {
pollStatus(); if (status && status.running) {
}) // Still running, poll again after short delay
}); setTimeout(pollStatus, 5000);
} else {
// Show loader if backend is still processing on page load elLoader.style.display = "none";
}
function pollStatus() { })
const elLoader = document.getElementById("loader"); .catch(() => {
fetch('/api/update-status') elLoader.style.display = "none";
.then(res => res.json()) });
}
.then(status => { pollStatus();
if (status && status.running) { })
elLoader.style.display = "inherit"; });
setTimeout(pollStatus, 5000);
} else { // Show loader if backend is still processing on page load
elLoader.style.display = "none";
} function pollStatus() {
}) const elLoader = document.getElementById("loader");
.catch(() => { fetch('/api/update-status')
elLoader.style.display = "none"; .then(res => res.json())
});
} .then(status => {
if (status && status.running) {
</script> elLoader.style.display = "inherit";
setTimeout(pollStatus, 5000);
</div> } else {
</body> elLoader.style.display = "none";
}
})
.catch(() => {
elLoader.style.display = "none";
});
}
</script>
</div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus(); // Immediate update
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
// Update bell appearance
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
// Update modal content
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
// Poll every 5 seconds
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
// Stop polling when page unloads
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html> </html>

308
js/add-class.js Normal file → Executable file
View File

@@ -1,154 +1,154 @@
export function addClass() { export function addClass() {
const input_class = document.querySelector('.add-category input.div-wrapper'); const input_class = document.querySelector('.add-category input.div-wrapper');
let existingClasses; let existingClasses;
const input_project_name = document.getElementById('project_name_input') const input_project_name = document.getElementById('project_name_input')
const description = document.getElementById('project_description_input'); const description = document.getElementById('project_description_input');
const button_addClass = document.querySelector('.add-category .upload-button-text-wrapper'); const button_addClass = document.querySelector('.add-category .upload-button-text-wrapper');
const button_addProject = document.querySelector('.popup .confirm-button-datasetcreation') const button_addProject = document.querySelector('.popup .confirm-button-datasetcreation')
const classWrapper = document.querySelector('.add-class-wrapper'); const classWrapper = document.querySelector('.add-class-wrapper');
button_addProject.addEventListener('click', () => { button_addProject.addEventListener('click', () => {
const title = input_project_name.value.trim(); const title = input_project_name.value.trim();
const descriptionText = description.value.trim(); const descriptionText = description.value.trim();
const classes = Array.from(classWrapper.querySelectorAll('.overlap-group')).map(el => el.textContent.trim()); const classes = Array.from(classWrapper.querySelectorAll('.overlap-group')).map(el => el.textContent.trim());
const formData = new FormData(); const formData = new FormData();
formData.append('title', title); formData.append('title', title);
formData.append('description', descriptionText); formData.append('description', descriptionText);
formData.append('classes', JSON.stringify(classes)); formData.append('classes', JSON.stringify(classes));
if (imgBlob) { if (imgBlob) {
formData.append('project_image', imgBlob, 'project_image.png'); // or the correct file type formData.append('project_image', imgBlob, 'project_image.png'); // or the correct file type
} }
fetch('/api/training-projects', { fetch('/api/training-projects', {
method: 'POST', method: 'POST',
body: formData body: formData
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert(data.message || 'Project created!'); alert(data.message || 'Project created!');
window.location.href = '/index.html'; window.location.href = '/index.html';
}) })
.catch(err => alert('Error: ' + err)); .catch(err => alert('Error: ' + err));
}); });
button_addClass.addEventListener('click', () => { button_addClass.addEventListener('click', () => {
const className = input_class.value.trim(); const className = input_class.value.trim();
if (!className) { if (!className) {
alert('Please enter a class name'); alert('Please enter a class name');
return; return;
} }
existingClasses = classWrapper.querySelectorAll('.overlap-group'); existingClasses = classWrapper.querySelectorAll('.overlap-group');
for (const el of existingClasses) { for (const el of existingClasses) {
if (el.textContent.trim().toLowerCase() === className.toLowerCase()) { if (el.textContent.trim().toLowerCase() === className.toLowerCase()) {
alert(`Class name "${className}" already exists.`); alert(`Class name "${className}" already exists.`);
return; return;
} }
} }
const newClassDiv = document.createElement('div'); const newClassDiv = document.createElement('div');
newClassDiv.classList.add('add-class'); newClassDiv.classList.add('add-class');
newClassDiv.style.position = 'relative'; newClassDiv.style.position = 'relative';
newClassDiv.style.width = '335px'; newClassDiv.style.width = '335px';
newClassDiv.style.height = '25px'; newClassDiv.style.height = '25px';
newClassDiv.style.marginBottom = '5px'; newClassDiv.style.marginBottom = '5px';
const overlapGroup = document.createElement('div'); const overlapGroup = document.createElement('div');
overlapGroup.classList.add('overlap-group'); overlapGroup.classList.add('overlap-group');
overlapGroup.style.position = 'absolute'; overlapGroup.style.position = 'absolute';
overlapGroup.style.width = '275px'; overlapGroup.style.width = '275px';
overlapGroup.style.height = '25px'; overlapGroup.style.height = '25px';
overlapGroup.style.top = '0'; overlapGroup.style.top = '0';
overlapGroup.style.left = '0'; overlapGroup.style.left = '0';
overlapGroup.style.backgroundColor = '#30bffc80'; overlapGroup.style.backgroundColor = '#30bffc80';
overlapGroup.style.borderRadius = '5px'; overlapGroup.style.borderRadius = '5px';
overlapGroup.style.display = 'flex'; overlapGroup.style.display = 'flex';
overlapGroup.style.alignItems = 'center'; overlapGroup.style.alignItems = 'center';
overlapGroup.style.paddingLeft = '10px'; overlapGroup.style.paddingLeft = '10px';
overlapGroup.style.color = '#000'; overlapGroup.style.color = '#000';
overlapGroup.style.fontFamily = 'var(--m3-body-small-font-family)'; overlapGroup.style.fontFamily = 'var(--m3-body-small-font-family)';
overlapGroup.style.fontWeight = 'var(--m3-body-small-font-weight)'; overlapGroup.style.fontWeight = 'var(--m3-body-small-font-weight)';
overlapGroup.style.fontSize = 'var(--m3-body-small-font-size)'; overlapGroup.style.fontSize = 'var(--m3-body-small-font-size)';
overlapGroup.style.letterSpacing = 'var(--m3-body-small-letter-spacing)'; overlapGroup.style.letterSpacing = 'var(--m3-body-small-letter-spacing)';
overlapGroup.style.lineHeight = 'var(--m3-body-small-line-height)'; overlapGroup.style.lineHeight = 'var(--m3-body-small-line-height)';
overlapGroup.textContent = className; overlapGroup.textContent = className;
const overlap = document.createElement('div'); const overlap = document.createElement('div');
overlap.classList.add('overlap'); overlap.classList.add('overlap');
overlap.style.position = 'absolute'; overlap.style.position = 'absolute';
overlap.style.width = '50px'; overlap.style.width = '50px';
overlap.style.height = '25px'; overlap.style.height = '25px';
overlap.style.top = '0'; overlap.style.top = '0';
overlap.style.left = '285px'; overlap.style.left = '285px';
const rectangle = document.createElement('div'); const rectangle = document.createElement('div');
rectangle.classList.add('rectangle'); rectangle.classList.add('rectangle');
rectangle.style.width = '50px'; rectangle.style.width = '50px';
rectangle.style.height = '25px'; rectangle.style.height = '25px';
rectangle.style.backgroundColor = '#ff0f43'; rectangle.style.backgroundColor = '#ff0f43';
rectangle.style.borderRadius = '5px'; rectangle.style.borderRadius = '5px';
rectangle.style.display = 'flex'; rectangle.style.display = 'flex';
rectangle.style.alignItems = 'center'; rectangle.style.alignItems = 'center';
rectangle.style.justifyContent = 'center'; rectangle.style.justifyContent = 'center';
rectangle.style.cursor = 'pointer'; rectangle.style.cursor = 'pointer';
rectangle.addEventListener('mouseenter', () => { rectangle.addEventListener('mouseenter', () => {
rectangle.style.backgroundColor = '#bb032b'; rectangle.style.backgroundColor = '#bb032b';
}); });
rectangle.addEventListener('mouseleave', () => { rectangle.addEventListener('mouseleave', () => {
rectangle.style.backgroundColor = '#ff0f43'; rectangle.style.backgroundColor = '#ff0f43';
}); });
const minusText = document.createElement('div'); const minusText = document.createElement('div');
minusText.classList.add('text-wrapper-4'); minusText.classList.add('text-wrapper-4');
minusText.style.position = 'absolute'; minusText.style.position = 'absolute';
minusText.style.top = '-18px'; minusText.style.top = '-18px';
minusText.style.left = '18px'; minusText.style.left = '18px';
minusText.style.fontFamily = 'var(--m3-display-large-font-family)'; minusText.style.fontFamily = 'var(--m3-display-large-font-family)';
minusText.style.fontWeight = 'var(--m3-display-large-font-weight)'; minusText.style.fontWeight = 'var(--m3-display-large-font-weight)';
minusText.style.color = '#000000'; minusText.style.color = '#000000';
minusText.style.fontSize = 'var(--minus-for-button-size)'; minusText.style.fontSize = 'var(--minus-for-button-size)';
minusText.style.letterSpacing = 'var(--m3-display-large-letter-spacing)'; minusText.style.letterSpacing = 'var(--m3-display-large-letter-spacing)';
minusText.style.lineHeight = 'var(--m3-display-large-line-height)'; minusText.style.lineHeight = 'var(--m3-display-large-line-height)';
minusText.style.whiteSpace = 'nowrap'; minusText.style.whiteSpace = 'nowrap';
minusText.style.cursor = 'pointer'; minusText.style.cursor = 'pointer';
minusText.style.fontStyle = 'var(--m3-display-large-font-style)'; minusText.style.fontStyle = 'var(--m3-display-large-font-style)';
minusText.textContent = '_'; minusText.textContent = '_';
rectangle.appendChild(minusText); rectangle.appendChild(minusText);
rectangle.addEventListener('click', () => { rectangle.addEventListener('click', () => {
classWrapper.removeChild(newClassDiv); classWrapper.removeChild(newClassDiv);
document.dispatchEvent(new CustomEvent('classListUpdated')); document.dispatchEvent(new CustomEvent('classListUpdated'));
}); });
overlap.appendChild(rectangle); overlap.appendChild(rectangle);
newClassDiv.appendChild(overlapGroup); newClassDiv.appendChild(overlapGroup);
newClassDiv.appendChild(overlap); newClassDiv.appendChild(overlap);
classWrapper.appendChild(newClassDiv); classWrapper.appendChild(newClassDiv);
input_class.value = ''; input_class.value = '';
}); });
} }

76
js/add-image.js Normal file → Executable file
View File

@@ -1,38 +1,38 @@
//Global Variable //Global Variable
var imgBlob; var imgBlob;
var imgMimeType var imgMimeType
// Create a hidden file input dynamically // Create a hidden file input dynamically
const fileInput = document.createElement('input'); const fileInput = document.createElement('input');
fileInput.type = 'file'; fileInput.type = 'file';
fileInput.accept = 'image/*'; fileInput.accept = 'image/*';
fileInput.style.display = 'none'; fileInput.style.display = 'none';
document.body.appendChild(fileInput); document.body.appendChild(fileInput);
function uploadButtonHandler() { function uploadButtonHandler() {
fileInput.click(); fileInput.click();
}; };
fileInput.addEventListener('change', () => { fileInput.addEventListener('change', () => {
const imageDiv = document.querySelector('.popup .image'); const imageDiv = document.querySelector('.popup .image');
const file = fileInput.files[0]; const file = fileInput.files[0];
if (!file) return; if (!file) return;
const reader = new FileReader(); const reader = new FileReader();
reader.onload = (e) => { reader.onload = (e) => {
imageDiv.innerHTML = ''; // clear previous content imageDiv.innerHTML = ''; // clear previous content
const img = document.createElement('img'); const img = document.createElement('img');
img.src = e.target.result; img.src = e.target.result;
img.alt = 'Uploaded Image'; img.alt = 'Uploaded Image';
img.style.width = '100%'; img.style.width = '100%';
img.style.height = '100%'; img.style.height = '100%';
img.style.objectFit = 'cover'; img.style.objectFit = 'cover';
img.style.borderRadius = '10px'; img.style.borderRadius = '10px';
imageDiv.appendChild(img); imageDiv.appendChild(img);
// Use the original file as the blob and store its MIME type // Use the original file as the blob and store its MIME type
imgBlob = file; imgBlob = file;
imgMimeType = file.type; imgMimeType = file.type;
}; };
reader.readAsDataURL(file); reader.readAsDataURL(file);
}); });

274
js/dashboard-label-studio.js Normal file → Executable file
View File

@@ -1,137 +1,137 @@
// Fetch LabelStudioProjects from backend and render as selectable cards // Fetch LabelStudioProjects from backend and render as selectable cards
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
let projectsList = document.getElementById('projects-list'); let projectsList = document.getElementById('projects-list');
const selectedIds = new Set(); const selectedIds = new Set();
if (!projectsList) { if (!projectsList) {
// Try to create the container if missing // Try to create the container if missing
projectsList = document.createElement('div'); projectsList = document.createElement('div');
projectsList.id = 'projects-list'; projectsList.id = 'projects-list';
document.body.appendChild(projectsList); document.body.appendChild(projectsList);
} }
else{console.log("noep")} else{console.log("noep")}
fetch('/api/label-studio-projects') fetch('/api/label-studio-projects')
.then(res => res.json()) .then(res => res.json())
.then(projects => { .then(projects => {
projectsList.innerHTML = ''; projectsList.innerHTML = '';
if (!projects || projects.length === 0) { if (!projects || projects.length === 0) {
projectsList.innerHTML = '<div>No Label Studio projects found</div>'; projectsList.innerHTML = '<div>No Label Studio projects found</div>';
return; return;
} }
for (const project of projects) { for (const project of projects) {
// Only show card if there is at least one non-empty annotation class // Only show card if there is at least one non-empty annotation class
const annotationClasses = Object.entries(project.annotationCounts || {}) const annotationClasses = Object.entries(project.annotationCounts || {})
.filter(([label, count]) => label && label.trim() !== ''); .filter(([label, count]) => label && label.trim() !== '');
if (annotationClasses.length === 0) continue; if (annotationClasses.length === 0) continue;
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.overflow = 'hidden'; card.style.overflow = 'hidden';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)';
card.style.display = 'flex'; card.style.display = 'flex';
card.style.background = 'white'; card.style.background = 'white';
card.style.cursor = 'pointer'; card.style.cursor = 'pointer';
card.tabIndex = 0; card.tabIndex = 0;
card.setAttribute('role', 'button'); card.setAttribute('role', 'button');
card.setAttribute('aria-label', `Open project ${project.title || project.project_id}`); card.setAttribute('aria-label', `Open project ${project.title || project.project_id}`);
// Selection logic // Selection logic
card.dataset.projectId = project.project_id; card.dataset.projectId = project.project_id;
card.addEventListener('click', () => { card.addEventListener('click', () => {
card.classList.toggle('selected'); card.classList.toggle('selected');
if (card.classList.contains('selected')) { if (card.classList.contains('selected')) {
card.style.background = '#009eac'; // main dif color for card card.style.background = '#009eac'; // main dif color for card
selectedIds.add(project.project_id); selectedIds.add(project.project_id);
} else { } else {
card.style.background = 'white'; // revert card color card.style.background = 'white'; // revert card color
selectedIds.delete(project.project_id); selectedIds.delete(project.project_id);
} }
// Debug: log selected ids array // Debug: log selected ids array
console.log(Array.from(selectedIds)); console.log(Array.from(selectedIds));
}); });
// Info // Info
const infoDiv = document.createElement('div'); const infoDiv = document.createElement('div');
infoDiv.className = 'info'; infoDiv.className = 'info';
infoDiv.style.background = 'rgba(210, 238, 240)'; infoDiv.style.background = 'rgba(210, 238, 240)';
infoDiv.style.flex = '1'; infoDiv.style.flex = '1';
infoDiv.style.padding = '16px'; infoDiv.style.padding = '16px';
infoDiv.innerHTML = ` infoDiv.innerHTML = `
<h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.project_id ?? 'N/A'} &nbsp;&nbsp;&nbsp; ${project.title || 'Untitled'}</h3> <h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.project_id ?? 'N/A'} &nbsp;&nbsp;&nbsp; ${project.title || 'Untitled'}</h3>
<div class="label-classes" style="font-size:1em;"> <div class="label-classes" style="font-size:1em;">
${annotationClasses.map(([label, count]) => `<p>${label}: ${count}</p>`).join('')} ${annotationClasses.map(([label, count]) => `<p>${label}: ${count}</p>`).join('')}
</div> </div>
`; `;
card.appendChild(infoDiv); card.appendChild(infoDiv);
projectsList.appendChild(card); projectsList.appendChild(card);
} }
}) })
.catch(() => { .catch(() => {
projectsList.innerHTML = '<div>Error loading Label Studio projects</div>'; projectsList.innerHTML = '<div>Error loading Label Studio projects</div>';
}); });
// Add Next button at the bottom right of the page // Add Next button at the bottom right of the page
const nextBtn = document.createElement('button'); const nextBtn = document.createElement('button');
nextBtn.id = 'next-btn'; nextBtn.id = 'next-btn';
nextBtn.className = 'button'; nextBtn.className = 'button';
nextBtn.textContent = 'Next'; nextBtn.textContent = 'Next';
nextBtn.style.position = 'fixed'; nextBtn.style.position = 'fixed';
nextBtn.style.right = '32px'; nextBtn.style.right = '32px';
nextBtn.style.bottom = '32px'; nextBtn.style.bottom = '32px';
nextBtn.style.zIndex = '1000'; nextBtn.style.zIndex = '1000';
document.body.appendChild(nextBtn); document.body.appendChild(nextBtn);
// Get training_project_id from URL // Get training_project_id from URL
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const trainingProjectId = urlParams.get('id'); const trainingProjectId = urlParams.get('id');
// Next button click handler // Next button click handler
nextBtn.addEventListener('click', () => { nextBtn.addEventListener('click', () => {
console.log(trainingProjectId) console.log(trainingProjectId)
if (!trainingProjectId) { if (!trainingProjectId) {
alert('No training project selected.'); alert('No training project selected.');
return; return;
} }
if (selectedIds.size === 0) { if (selectedIds.size === 0) {
alert('Please select at least one Label Studio project.'); alert('Please select at least one Label Studio project.');
return; return;
} }
const annotationProjectsJson = JSON.stringify(Array.from(selectedIds)); const annotationProjectsJson = JSON.stringify(Array.from(selectedIds));
fetch('/api/training-project-details', { fetch('/api/training-project-details', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ body: JSON.stringify({
project_id: Number(trainingProjectId), project_id: Number(trainingProjectId),
annotation_projects: Array.from(selectedIds) annotation_projects: Array.from(selectedIds)
}) })
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert('TrainingProjectDetails saved!'); alert('TrainingProjectDetails saved!');
console.log(data); console.log(data);
// Redirect to start-training.html with id // Redirect to start-training.html with id
window.location.href = `/setup-training-project.html?id=${trainingProjectId}`; window.location.href = `/setup-training-project.html?id=${trainingProjectId}`;
}) })
.catch(err => { .catch(err => {
alert('Error saving TrainingProjectDetails'); alert('Error saving TrainingProjectDetails');
console.error(err); console.error(err);
}); });
}); });
// Add description field above the project cards // Add description field above the project cards
const descDiv = document.createElement('div'); const descDiv = document.createElement('div');
descDiv.id = 'dashboard-description'; descDiv.id = 'dashboard-description';
descDiv.style.width = '100%'; descDiv.style.width = '100%';
descDiv.style.maxWidth = '900px'; descDiv.style.maxWidth = '900px';
descDiv.style.margin = '0 auto 24px auto'; descDiv.style.margin = '0 auto 24px auto';
descDiv.style.padding = '18px 24px'; descDiv.style.padding = '18px 24px';
descDiv.style.background = '#eaf7fa'; descDiv.style.background = '#eaf7fa';
descDiv.style.borderRadius = '12px'; descDiv.style.borderRadius = '12px';
descDiv.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)'; descDiv.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)';
descDiv.style.fontSize = '1.15em'; descDiv.style.fontSize = '1.15em';
descDiv.style.color = '#009eac'; descDiv.style.color = '#009eac';
descDiv.style.textAlign = 'center'; descDiv.style.textAlign = 'center';
descDiv.textContent = 'Select one or more Label Studio projects by clicking the cards below. The annotation summary for each project is shown. Click Next to continue.'; descDiv.textContent = 'Select one or more Label Studio projects by clicking the cards below. The annotation summary for each project is shown. Click Next to continue.';
projectsList.parentNode.insertBefore(descDiv, projectsList); projectsList.parentNode.insertBefore(descDiv, projectsList);
}); });

342
js/dashboard.js Normal file → Executable file
View File

@@ -1,171 +1,171 @@
function renderProjects(projects) { function renderProjects(projects) {
const projectsList = document.getElementById('projects-list'); const projectsList = document.getElementById('projects-list');
projectsList.innerHTML = ''; projectsList.innerHTML = '';
if (projects.length === 0) { if (projects.length === 0) {
projectsList.innerHTML = '<div>No projects found</div>'; projectsList.innerHTML = '<div>No projects found</div>';
return; return;
} }
for (const project of projects) { for (const project of projects) {
const labelCounts = project.labelCounts || {}; const labelCounts = project.labelCounts || {};
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.overflow = 'hidden'; card.style.overflow = 'hidden';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0)';
card.style.display = 'flex'; card.style.display = 'flex';
card.style.background = 'white'; card.style.background = 'white';
card.style.cursor = 'pointer'; card.style.cursor = 'pointer';
card.tabIndex = 0; card.tabIndex = 0;
card.setAttribute('role', 'button'); card.setAttribute('role', 'button');
card.setAttribute('aria-label', `Open project ${project.title || project.id}`); card.setAttribute('aria-label', `Open project ${project.title || project.id}`);
card.style.position = 'relative'; // For absolute positioning of delete button card.style.position = 'relative'; // For absolute positioning of delete button
card.addEventListener('click', (e) => { card.addEventListener('click', (e) => {
// Prevent click if delete button is pressed // Prevent click if delete button is pressed
if (e.target.classList.contains('delete-btn')) return; if (e.target.classList.contains('delete-btn')) return;
if (project.hasTraining) { if (project.hasTraining) {
window.location.href = `/overview-training.html?id=${project.id}`; window.location.href = `/overview-training.html?id=${project.id}`;
} else if (project.hasDetails) { } else if (project.hasDetails) {
// Find details for this project // Find details for this project
const detailsEntry = window._trainingProjectDetails?.find(d => d.project_id == project.id); const detailsEntry = window._trainingProjectDetails?.find(d => d.project_id == project.id);
if (detailsEntry && Array.isArray(detailsEntry.class_map) && detailsEntry.class_map.length > 0) { if (detailsEntry && Array.isArray(detailsEntry.class_map) && detailsEntry.class_map.length > 0) {
// If classes are assigned, skip to start-training.html // If classes are assigned, skip to start-training.html
window.location.href = `/edit-training.html?id=${project.id}`; window.location.href = `/edit-training.html?id=${project.id}`;
} else { } else {
window.location.href = `/setup-training-project.html?id=${project.id}`; window.location.href = `/setup-training-project.html?id=${project.id}`;
} }
} else { } else {
window.location.href = `/project-details.html?id=${project.id}`; window.location.href = `/project-details.html?id=${project.id}`;
} }
}); });
// Image // Image
let imageHTML = ''; let imageHTML = '';
if (project.project_image) { if (project.project_image) {
imageHTML = `<img src="${project.project_image}" alt="img" style="width:120px;height:120px;object-fit:cover;display:block;" />`; imageHTML = `<img src="${project.project_image}" alt="img" style="width:120px;height:120px;object-fit:cover;display:block;" />`;
} }
const imgContainer = document.createElement('div'); const imgContainer = document.createElement('div');
imgContainer.className = 'img-container'; imgContainer.className = 'img-container';
imgContainer.style.background = '#009eac2d' imgContainer.style.background = '#009eac2d'
imgContainer.style.flex = '0 0 120px'; imgContainer.style.flex = '0 0 120px';
imgContainer.style.display = 'flex'; imgContainer.style.display = 'flex';
imgContainer.style.alignItems = 'center'; imgContainer.style.alignItems = 'center';
imgContainer.style.justifyContent = 'center'; imgContainer.style.justifyContent = 'center';
imgContainer.innerHTML = imageHTML; imgContainer.innerHTML = imageHTML;
// Info // Info
const infoDiv = document.createElement('div'); const infoDiv = document.createElement('div');
infoDiv.className = 'info'; infoDiv.className = 'info';
infoDiv.style.background = '#009eac2d' infoDiv.style.background = '#009eac2d'
infoDiv.style.flex = '1'; infoDiv.style.flex = '1';
infoDiv.style.padding = '16px'; infoDiv.style.padding = '16px';
infoDiv.innerHTML = ` infoDiv.innerHTML = `
<h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.id ?? 'N/A'} &nbsp&nbsp&nbsp ${project.title || 'Untitled'}</h3> <h3 style="margin:0 0 8px 0;font-size:1.5em;font-weight:bold;">${project.id ?? 'N/A'} &nbsp&nbsp&nbsp ${project.title || 'Untitled'}</h3>
<div class="label-classes" style="font-size:1em;"> <div class="label-classes" style="font-size:1em;">
${getClassesAsParagraphs(project, labelCounts)} ${getClassesAsParagraphs(project, labelCounts)}
</div> </div>
`; `;
// Delete button // Delete button
const deleteBtn = document.createElement('button'); const deleteBtn = document.createElement('button');
deleteBtn.textContent = 'Delete'; deleteBtn.textContent = 'Delete';
deleteBtn.style.width = '70px'; deleteBtn.style.width = '70px';
deleteBtn.style.height = '28px'; deleteBtn.style.height = '28px';
deleteBtn.className = 'button-red delete-btn'; deleteBtn.className = 'button-red delete-btn';
deleteBtn.style.position = 'absolute'; deleteBtn.style.position = 'absolute';
deleteBtn.style.bottom = '0px'; deleteBtn.style.bottom = '0px';
deleteBtn.style.right = '15px'; deleteBtn.style.right = '15px';
deleteBtn.style.zIndex = '2'; deleteBtn.style.zIndex = '2';
deleteBtn.style.fontSize = '14px'; deleteBtn.style.fontSize = '14px';
deleteBtn.style.padding = '0'; deleteBtn.style.padding = '0';
deleteBtn.style.borderRadius = '6px'; deleteBtn.style.borderRadius = '6px';
deleteBtn.style.boxShadow = '0 2px 8px rgba(0,0,0,0.08)'; deleteBtn.style.boxShadow = '0 2px 8px rgba(0,0,0,0.08)';
deleteBtn.style.display = 'flex'; deleteBtn.style.display = 'flex';
deleteBtn.style.alignItems = 'center'; deleteBtn.style.alignItems = 'center';
deleteBtn.style.justifyContent = 'center'; deleteBtn.style.justifyContent = 'center';
deleteBtn.addEventListener('click', function(e) { deleteBtn.addEventListener('click', function(e) {
e.stopPropagation(); e.stopPropagation();
if (confirm('Are you sure you want to delete this training project?')) { if (confirm('Are you sure you want to delete this training project?')) {
fetch(`/api/training-projects/${project.id}`, { method: 'DELETE' }) fetch(`/api/training-projects/${project.id}`, { method: 'DELETE' })
.then(res => { .then(res => {
if (res.ok) { if (res.ok) {
card.remove(); card.remove();
} else { } else {
alert('Failed to delete project.'); alert('Failed to delete project.');
} }
}) })
.catch(() => alert('Failed to delete project.')); .catch(() => alert('Failed to delete project.'));
} }
}); });
card.appendChild(imgContainer); card.appendChild(imgContainer);
card.appendChild(infoDiv); card.appendChild(infoDiv);
card.appendChild(deleteBtn); card.appendChild(deleteBtn);
projectsList.appendChild(card); projectsList.appendChild(card);
} }
} }
// Helper to render classes as <p> elements // Helper to render classes as <p> elements
function getClassesAsParagraphs(project, labelCounts) { function getClassesAsParagraphs(project, labelCounts) {
let classes = []; let classes = [];
let labelConfig = project.parsed_label_config; let labelConfig = project.parsed_label_config;
if (typeof labelConfig === 'string') { if (typeof labelConfig === 'string') {
try { labelConfig = JSON.parse(labelConfig); } catch { labelConfig = null; } try { labelConfig = JSON.parse(labelConfig); } catch { labelConfig = null; }
} }
if (labelConfig) { if (labelConfig) {
Object.values(labelConfig).forEach(cfg => { Object.values(labelConfig).forEach(cfg => {
if (cfg.labels && Array.isArray(cfg.labels)) { if (cfg.labels && Array.isArray(cfg.labels)) {
cfg.labels.forEach(label => { cfg.labels.forEach(label => {
classes.push(label); classes.push(label);
}); });
} }
}); });
} }
if (classes.length === 0 && project.prompts && project.prompts.length > 0) { if (classes.length === 0 && project.prompts && project.prompts.length > 0) {
const prompt = project.prompts[0]; const prompt = project.prompts[0];
if (prompt.output_classes && prompt.output_classes.length > 0) { if (prompt.output_classes && prompt.output_classes.length > 0) {
classes = prompt.output_classes; classes = prompt.output_classes;
} }
} }
if (classes.length === 0 && Object.keys(labelCounts).length > 0) { if (classes.length === 0 && Object.keys(labelCounts).length > 0) {
classes = Object.keys(labelCounts); classes = Object.keys(labelCounts);
} }
return classes.map(cls => `<p>${cls}${labelCounts && labelCounts[cls] !== undefined ? ' ' : ''}</p>`).join(''); return classes.map(cls => `<p>${cls}${labelCounts && labelCounts[cls] !== undefined ? ' ' : ''}</p>`).join('');
} }
// Fetch and render TrainingProjects from the backend // Fetch and render TrainingProjects from the backend
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
Promise.all([ Promise.all([
fetch('/api/training-projects').then(res => res.json()), fetch('/api/training-projects').then(res => res.json()),
fetch('/api/training-project-details').then(res => res.json()), fetch('/api/training-project-details').then(res => res.json()),
fetch('/api/trainings').then(res => res.json()) fetch('/api/trainings').then(res => res.json())
]).then(([projects, details, trainings]) => { ]).then(([projects, details, trainings]) => {
window._trainingProjectDetails = details; // Store globally for click handler window._trainingProjectDetails = details; // Store globally for click handler
// Build a set of project IDs that have details // Build a set of project IDs that have details
const detailsProjectIds = new Set(details.map(d => d.project_id)); const detailsProjectIds = new Set(details.map(d => d.project_id));
// Build a set of project IDs that have trainings // Build a set of project IDs that have trainings
const detailsIdToProjectId = {}; const detailsIdToProjectId = {};
details.forEach(d => { detailsIdToProjectId[d.id] = d.project_id; }); details.forEach(d => { detailsIdToProjectId[d.id] = d.project_id; });
const trainingProjectIds = new Set(trainings.map(t => detailsIdToProjectId[t.project_details_id])); const trainingProjectIds = new Set(trainings.map(t => detailsIdToProjectId[t.project_details_id]));
// Map project_id to id for frontend compatibility // Map project_id to id for frontend compatibility
projects.forEach(project => { projects.forEach(project => {
if (project.project_id !== undefined) project.id = project.project_id; if (project.project_id !== undefined) project.id = project.project_id;
if (Array.isArray(project.classes)) { if (Array.isArray(project.classes)) {
project.labelCounts = {}; project.labelCounts = {};
project.classes.forEach(cls => project.labelCounts[cls] = 0); project.classes.forEach(cls => project.labelCounts[cls] = 0);
} }
// Attach a flag for details existence // Attach a flag for details existence
project.hasDetails = detailsProjectIds.has(project.id); project.hasDetails = detailsProjectIds.has(project.id);
// Attach a flag for training existence // Attach a flag for training existence
project.hasTraining = trainingProjectIds.has(project.id); project.hasTraining = trainingProjectIds.has(project.id);
}); });
renderProjects(projects); renderProjects(projects);
}).catch(err => { }).catch(err => {
document.getElementById('projects-list').innerHTML = '<div>Error loading projects</div>'; document.getElementById('projects-list').innerHTML = '<div>Error loading projects</div>';
}); });
}); });

0
js/overview-training.js Normal file → Executable file
View File

224
js/settings.js Executable file
View File

@@ -0,0 +1,224 @@
// Settings Modal Management
// Function to open modal
window.openSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'flex';
loadSettings();
}
};
// Function to close modal
window.closeSettingsModal = function() {
const modal = document.getElementById('settings-modal');
if (modal) {
modal.style.display = 'none';
}
};
// Close modal when clicking outside
window.addEventListener('click', function(event) {
const modal = document.getElementById('settings-modal');
if (event.target === modal) {
window.closeSettingsModal();
}
});
// Load settings when modal opens
async function loadSettings() {
try {
const response = await fetch('/api/settings');
if (!response.ok) {
throw new Error('Failed to load settings');
}
const settings = await response.json();
const settingsMap = {};
settings.forEach(s => {
settingsMap[s.key] = s.value;
});
// Populate fields with correct IDs
const labelstudioUrl = document.getElementById('labelstudio-url');
const labelstudioToken = document.getElementById('labelstudio-token');
const yoloxPathInput = document.getElementById('yolox-path');
const yoloxVenvPathInput = document.getElementById('yolox-venv-path');
const yoloxOutputPathInput = document.getElementById('yolox-output-path');
const yoloxDataDirInput = document.getElementById('yolox-data-dir');
if (labelstudioUrl) labelstudioUrl.value = settingsMap.labelstudio_api_url || '';
if (labelstudioToken) labelstudioToken.value = settingsMap.labelstudio_api_token || '';
if (yoloxPathInput) yoloxPathInput.value = settingsMap.yolox_path || '';
if (yoloxVenvPathInput) yoloxVenvPathInput.value = settingsMap.yolox_venv_path || '';
if (yoloxOutputPathInput) yoloxOutputPathInput.value = settingsMap.yolox_output_path || '';
if (yoloxDataDirInput) yoloxDataDirInput.value = settingsMap.yolox_data_dir || '';
} catch (error) {
console.error('Error loading settings:', error);
const saveStatus = document.getElementById('save-status');
if (saveStatus) {
showMessage(saveStatus, 'Fehler beim Laden: ' + error.message, 'error');
}
}
}
// Helper functions
function showMessage(element, message, type) {
if (element) {
element.textContent = message;
element.className = 'status-message ' + type;
element.style.display = 'block';
}
}
function hideMessage(element) {
if (element) {
element.style.display = 'none';
}
}
// Event listeners - wait for DOM to be ready
document.addEventListener('DOMContentLoaded', function() {
// Test Label Studio connection
const testLabelStudioBtn = document.getElementById('test-labelstudio-btn');
if (testLabelStudioBtn) {
testLabelStudioBtn.addEventListener('click', async () => {
const apiUrl = document.getElementById('labelstudio-url').value.trim();
const apiToken = document.getElementById('labelstudio-token').value.trim();
const labelstudioStatus = document.getElementById('labelstudio-status');
const loader = document.getElementById('test-ls-loader');
if (!apiUrl || !apiToken) {
showMessage(labelstudioStatus, 'Bitte geben Sie URL und Token ein', 'error');
return;
}
testLabelStudioBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(labelstudioStatus);
try {
const response = await fetch('/api/settings/test/labelstudio', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ api_url: apiUrl, api_token: apiToken })
});
const result = await response.json();
if (result.success) {
showMessage(labelstudioStatus, '✓ ' + result.message, 'success');
} else {
showMessage(labelstudioStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(labelstudioStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testLabelStudioBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Test YOLOX path
const testYoloxBtn = document.getElementById('test-yolox-btn');
if (testYoloxBtn) {
testYoloxBtn.addEventListener('click', async () => {
const path = document.getElementById('yolox-path').value.trim();
const venvPath = document.getElementById('yolox-venv-path').value.trim();
const yoloxStatus = document.getElementById('yolox-status');
const loader = document.getElementById('test-yolox-loader');
if (!path) {
showMessage(yoloxStatus, 'Bitte geben Sie einen YOLOX Pfad ein', 'error');
return;
}
testYoloxBtn.disabled = true;
if (loader) loader.style.display = 'block';
hideMessage(yoloxStatus);
try {
const response = await fetch('/api/settings/test/yolox', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
yolox_path: path,
yolox_venv_path: venvPath
})
});
const result = await response.json();
if (result.success) {
showMessage(yoloxStatus, '✓ ' + result.message, 'success');
} else {
showMessage(yoloxStatus, '✗ ' + result.message, 'error');
}
} catch (error) {
showMessage(yoloxStatus, '✗ Fehler: ' + error.message, 'error');
} finally {
testYoloxBtn.disabled = false;
if (loader) loader.style.display = 'none';
}
});
}
// Save settings
const saveSettingsBtn = document.getElementById('save-settings-btn');
if (saveSettingsBtn) {
saveSettingsBtn.addEventListener('click', async () => {
const labelstudioUrl = document.getElementById('labelstudio-url').value.trim();
const labelstudioToken = document.getElementById('labelstudio-token').value.trim();
const yoloxPathValue = document.getElementById('yolox-path').value.trim();
const yoloxVenvPathValue = document.getElementById('yolox-venv-path').value.trim();
const yoloxOutputPathValue = document.getElementById('yolox-output-path').value.trim();
const yoloxDataDirValue = document.getElementById('yolox-data-dir').value.trim();
const saveStatus = document.getElementById('save-status');
// Validation
if (!labelstudioUrl || !labelstudioToken || !yoloxPathValue || !yoloxVenvPathValue || !yoloxOutputPathValue || !yoloxDataDirValue) {
showMessage(saveStatus, 'Bitte füllen Sie alle Felder aus', 'error');
return;
}
const settings = {
labelstudio_api_url: labelstudioUrl,
labelstudio_api_token: labelstudioToken,
yolox_path: yoloxPathValue,
yolox_venv_path: yoloxVenvPathValue,
yolox_output_path: yoloxOutputPathValue,
yolox_data_dir: yoloxDataDirValue
};
saveSettingsBtn.disabled = true;
const originalText = saveSettingsBtn.textContent;
saveSettingsBtn.textContent = 'Speichern...';
hideMessage(saveStatus);
try {
const response = await fetch('/api/settings', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings)
});
if (!response.ok) {
throw new Error('Failed to save settings');
}
showMessage(saveStatus, '✓ Einstellungen erfolgreich gespeichert!', 'success');
// Close modal after 1.5 seconds
setTimeout(() => {
window.closeSettingsModal();
}, 1500);
} catch (error) {
showMessage(saveStatus, '✗ Fehler beim Speichern: ' + error.message, 'error');
} finally {
saveSettingsBtn.disabled = false;
saveSettingsBtn.textContent = originalText;
}
});
}
});

430
js/setup-training-project.js Normal file → Executable file
View File

@@ -1,216 +1,216 @@
// Fetch and display training project name in nav bar // Fetch and display training project name in nav bar
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const trainingProjectId = urlParams.get('id'); const trainingProjectId = urlParams.get('id');
if (!trainingProjectId) return; if (!trainingProjectId) return;
// Fetch training project, details, and all LabelStudioProjects // Fetch training project, details, and all LabelStudioProjects
Promise.all([ Promise.all([
fetch(`/api/training-projects`).then(res => res.json()), fetch(`/api/training-projects`).then(res => res.json()),
fetch(`/api/training-project-details`).then(res => res.json()), fetch(`/api/training-project-details`).then(res => res.json()),
fetch(`/api/label-studio-projects`).then(res => res.json()) fetch(`/api/label-studio-projects`).then(res => res.json())
]).then(([projects, detailsList, labelStudioProjects]) => { ]).then(([projects, detailsList, labelStudioProjects]) => {
// Find the selected training project // Find the selected training project
const project = projects.find(p => p.project_id == trainingProjectId || p.id == trainingProjectId); const project = projects.find(p => p.project_id == trainingProjectId || p.id == trainingProjectId);
// Find the details entry for this project // Find the details entry for this project
const details = detailsList.find(d => d.project_id == trainingProjectId); const details = detailsList.find(d => d.project_id == trainingProjectId);
if (!project || !details) return; if (!project || !details) return;
// Get the stored classes from training project // Get the stored classes from training project
const storedClasses = Array.isArray(project.classes) ? project.classes : []; const storedClasses = Array.isArray(project.classes) ? project.classes : [];
// Get related LabelStudioProject IDs // Get related LabelStudioProject IDs
const relatedIds = Array.isArray(details.annotation_projects) ? details.annotation_projects : []; const relatedIds = Array.isArray(details.annotation_projects) ? details.annotation_projects : [];
// Filter LabelStudioProjects to only those related // Filter LabelStudioProjects to only those related
const relatedProjects = labelStudioProjects.filter(lp => relatedIds.includes(lp.project_id)); const relatedProjects = labelStudioProjects.filter(lp => relatedIds.includes(lp.project_id));
// Render cards for each related LabelStudioProject // Render cards for each related LabelStudioProject
const detailsDiv = document.getElementById('details'); const detailsDiv = document.getElementById('details');
detailsDiv.innerHTML = ''; detailsDiv.innerHTML = '';
// Find the longest label name for sizing // Find the longest label name for sizing
let maxLabelLength = 0; let maxLabelLength = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
classNames.forEach(className => { classNames.forEach(className => {
if (className && className.trim() !== '' && className.length > maxLabelLength) { if (className && className.trim() !== '' && className.length > maxLabelLength) {
maxLabelLength = className.length; maxLabelLength = className.length;
} }
}); });
}); });
// Use ch unit for width to fit the longest text // Use ch unit for width to fit the longest text
const labelWidth = `${maxLabelLength + 2}ch`; const labelWidth = `${maxLabelLength + 2}ch`;
// Find the longest project name for sizing // Find the longest project name for sizing
let maxProjectNameLength = 0; let maxProjectNameLength = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const nameLength = (lp.title || String(lp.project_id)).length; const nameLength = (lp.title || String(lp.project_id)).length;
if (nameLength > maxProjectNameLength) maxProjectNameLength = nameLength; if (nameLength > maxProjectNameLength) maxProjectNameLength = nameLength;
}); });
const projectNameWidth = `${maxProjectNameLength + 2}ch`; const projectNameWidth = `${maxProjectNameLength + 2}ch`;
// Find the card with the most classes // Find the card with the most classes
let maxClassCount = 0; let maxClassCount = 0;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
if (classNames.length > maxClassCount) maxClassCount = classNames.length; if (classNames.length > maxClassCount) maxClassCount = classNames.length;
}); });
// Set a fixed width for the class rows container // Set a fixed width for the class rows container
const classRowHeight = 38; // px, adjust if needed const classRowHeight = 38; // px, adjust if needed
const classRowsWidth = `${maxClassCount * 180}px`; const classRowsWidth = `${maxClassCount * 180}px`;
relatedProjects.forEach(lp => { relatedProjects.forEach(lp => {
// Get original class names from annotationCounts // Get original class names from annotationCounts
const classNames = Object.keys(lp.annotationCounts || {}); const classNames = Object.keys(lp.annotationCounts || {});
const card = document.createElement('div'); const card = document.createElement('div');
card.className = 'card'; card.className = 'card';
card.style.margin = '18px 0'; card.style.margin = '18px 0';
card.style.padding = '18px'; card.style.padding = '18px';
card.style.borderRadius = '12px'; card.style.borderRadius = '12px';
card.style.background = '#f5f5f5'; card.style.background = '#f5f5f5';
card.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)'; card.style.boxShadow = '0 2px 8px rgba(0,0,0,0.04)';
// Extra div for project name // Extra div for project name
const nameDiv = document.createElement('div'); const nameDiv = document.createElement('div');
nameDiv.textContent = lp.title || lp.project_id; nameDiv.textContent = lp.title || lp.project_id;
nameDiv.style.fontSize = '1.2em'; nameDiv.style.fontSize = '1.2em';
nameDiv.style.fontWeight = 'bold'; nameDiv.style.fontWeight = 'bold';
nameDiv.style.marginBottom = '12px'; nameDiv.style.marginBottom = '12px';
nameDiv.style.background = '#eaf7fa'; nameDiv.style.background = '#eaf7fa';
nameDiv.style.padding = '8px 16px'; nameDiv.style.padding = '8px 16px';
nameDiv.style.borderRadius = '8px'; nameDiv.style.borderRadius = '8px';
nameDiv.style.width = projectNameWidth; nameDiv.style.width = projectNameWidth;
nameDiv.style.minWidth = projectNameWidth; nameDiv.style.minWidth = projectNameWidth;
nameDiv.style.maxWidth = projectNameWidth; nameDiv.style.maxWidth = projectNameWidth;
nameDiv.style.display = 'inline-block'; nameDiv.style.display = 'inline-block';
card.appendChild(nameDiv); card.appendChild(nameDiv);
// Container for class rows // Container for class rows
const classRowsDiv = document.createElement('div'); const classRowsDiv = document.createElement('div');
classRowsDiv.style.display = 'inline-block'; classRowsDiv.style.display = 'inline-block';
classRowsDiv.style.verticalAlign = 'top'; classRowsDiv.style.verticalAlign = 'top';
classRowsDiv.style.width = classRowsWidth; classRowsDiv.style.width = classRowsWidth;
classNames.forEach(className => { classNames.forEach(className => {
// Row for class name and dropdown // Row for class name and dropdown
const row = document.createElement('div'); const row = document.createElement('div');
row.className = 'class-row'; // Mark as class row row.className = 'class-row'; // Mark as class row
row.style.display = 'flex'; row.style.display = 'flex';
row.style.alignItems = 'center'; row.style.alignItems = 'center';
row.style.marginBottom = '10px'; row.style.marginBottom = '10px';
// Original class name // Original class name
const labelSpan = document.createElement('span'); const labelSpan = document.createElement('span');
labelSpan.textContent = className; labelSpan.textContent = className;
labelSpan.style.fontWeight = 'bold'; labelSpan.style.fontWeight = 'bold';
labelSpan.style.marginRight = '16px'; labelSpan.style.marginRight = '16px';
labelSpan.style.width = labelWidth; labelSpan.style.width = labelWidth;
labelSpan.style.minWidth = labelWidth; labelSpan.style.minWidth = labelWidth;
labelSpan.style.maxWidth = labelWidth; labelSpan.style.maxWidth = labelWidth;
labelSpan.style.display = 'inline-block'; labelSpan.style.display = 'inline-block';
// Dropdown for reassigning // Dropdown for reassigning
const select = document.createElement('select'); const select = document.createElement('select');
select.style.marginLeft = '8px'; select.style.marginLeft = '8px';
select.style.padding = '4px 8px'; select.style.padding = '4px 8px';
select.style.borderRadius = '6px'; select.style.borderRadius = '6px';
select.style.border = '1px solid #009eac'; select.style.border = '1px solid #009eac';
// Add blank item // Add blank item
const blankOption = document.createElement('option'); const blankOption = document.createElement('option');
blankOption.value = ''; blankOption.value = '';
blankOption.textContent = ''; blankOption.textContent = '';
select.appendChild(blankOption); select.appendChild(blankOption);
storedClasses.forEach(cls => { storedClasses.forEach(cls => {
const option = document.createElement('option'); const option = document.createElement('option');
option.value = cls; option.value = cls;
option.textContent = cls; option.textContent = cls;
select.appendChild(option); select.appendChild(option);
}); });
row.appendChild(labelSpan); row.appendChild(labelSpan);
row.appendChild(select); row.appendChild(select);
classRowsDiv.appendChild(row); classRowsDiv.appendChild(row);
}); });
card.appendChild(classRowsDiv); card.appendChild(classRowsDiv);
// Description field (right side, last element) // Description field (right side, last element)
const descDiv = document.createElement('div'); const descDiv = document.createElement('div');
descDiv.className = 'card-description'; descDiv.className = 'card-description';
descDiv.style.flex = '1'; descDiv.style.flex = '1';
descDiv.style.marginLeft = '32px'; descDiv.style.marginLeft = '32px';
descDiv.style.display = 'flex'; descDiv.style.display = 'flex';
descDiv.style.flexDirection = 'column'; descDiv.style.flexDirection = 'column';
descDiv.style.justifyContent = 'flex-start'; descDiv.style.justifyContent = 'flex-start';
descDiv.style.alignItems = 'flex-start'; descDiv.style.alignItems = 'flex-start';
descDiv.style.width = '220px'; descDiv.style.width = '220px';
// Add a label and textarea for description // Add a label and textarea for description
const descLabel = document.createElement('label'); const descLabel = document.createElement('label');
descLabel.textContent = 'Description:'; descLabel.textContent = 'Description:';
descLabel.style.fontWeight = 'bold'; descLabel.style.fontWeight = 'bold';
descLabel.style.marginBottom = '4px'; descLabel.style.marginBottom = '4px';
const descTextarea = document.createElement('textarea'); const descTextarea = document.createElement('textarea');
descTextarea.style.width = '220px'; descTextarea.style.width = '220px';
descTextarea.style.height = '48px'; descTextarea.style.height = '48px';
descTextarea.style.borderRadius = '6px'; descTextarea.style.borderRadius = '6px';
descTextarea.style.border = '1px solid #009eac'; descTextarea.style.border = '1px solid #009eac';
descTextarea.style.padding = '6px'; descTextarea.style.padding = '6px';
descTextarea.style.resize = 'none'; descTextarea.style.resize = 'none';
descTextarea.value = lp.description || ''; descTextarea.value = lp.description || '';
descDiv.appendChild(descLabel); descDiv.appendChild(descLabel);
descDiv.appendChild(descTextarea); descDiv.appendChild(descTextarea);
card.appendChild(descDiv); card.appendChild(descDiv);
detailsDiv.appendChild(card); detailsDiv.appendChild(card);
}); });
// Add Next button at the bottom right of the page // Add Next button at the bottom right of the page
const nextBtn = document.createElement('button'); const nextBtn = document.createElement('button');
nextBtn.id = 'next-btn'; nextBtn.id = 'next-btn';
nextBtn.className = 'button'; nextBtn.className = 'button';
nextBtn.textContent = 'Next'; nextBtn.textContent = 'Next';
nextBtn.style.position = 'fixed'; nextBtn.style.position = 'fixed';
nextBtn.style.right = '32px'; nextBtn.style.right = '32px';
nextBtn.style.bottom = '32px'; nextBtn.style.bottom = '32px';
nextBtn.style.zIndex = '1000'; nextBtn.style.zIndex = '1000';
document.body.appendChild(nextBtn); document.body.appendChild(nextBtn);
// Next button click handler: collect class mappings and update TrainingProjectDetails // Next button click handler: collect class mappings and update TrainingProjectDetails
nextBtn.addEventListener('click', () => { nextBtn.addEventListener('click', () => {
// Array of arrays: [[labelStudioProjectId, [[originalClass, mappedClass], ...]], ...] // Array of arrays: [[labelStudioProjectId, [[originalClass, mappedClass], ...]], ...]
const mappings = []; const mappings = [];
const descriptions = []; const descriptions = [];
detailsDiv.querySelectorAll('.card').forEach((card, idx) => { detailsDiv.querySelectorAll('.card').forEach((card, idx) => {
const projectId = relatedProjects[idx].project_id; const projectId = relatedProjects[idx].project_id;
const classMap = []; const classMap = [];
// Only iterate over actual class rows // Only iterate over actual class rows
card.querySelectorAll('.class-row').forEach(row => { card.querySelectorAll('.class-row').forEach(row => {
const labelSpan = row.querySelector('span'); const labelSpan = row.querySelector('span');
const select = row.querySelector('select'); const select = row.querySelector('select');
if (labelSpan && select) { if (labelSpan && select) {
const className = labelSpan.textContent.trim(); const className = labelSpan.textContent.trim();
const mappedValue = select.value.trim(); const mappedValue = select.value.trim();
if (className !== '' && mappedValue !== '') { if (className !== '' && mappedValue !== '') {
classMap.push([className, mappedValue]); classMap.push([className, mappedValue]);
} }
} }
}); });
mappings.push([projectId, classMap]); mappings.push([projectId, classMap]);
// Get description from textarea // Get description from textarea
const descTextarea = card.querySelector('textarea'); const descTextarea = card.querySelector('textarea');
descriptions.push([projectId, descTextarea ? descTextarea.value : '']); descriptions.push([projectId, descTextarea ? descTextarea.value : '']);
}); });
// Update TrainingProjectDetails in DB // Update TrainingProjectDetails in DB
fetch('/api/training-project-details', { fetch('/api/training-project-details', {
method: 'PUT', method: 'PUT',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ body: JSON.stringify({
project_id: Number(trainingProjectId), project_id: Number(trainingProjectId),
class_map: mappings, class_map: mappings,
description: descriptions // array of [projectId, description] description: descriptions // array of [projectId, description]
}) })
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
alert('Class assignments and descriptions updated!'); alert('Class assignments and descriptions updated!');
console.log(data); console.log(data);
// Redirect to start-training.html with id // Redirect to start-training.html with id
window.location.href = `/edit-training.html?id=${trainingProjectId}`; window.location.href = `/edit-training.html?id=${trainingProjectId}`;
}) })
.catch(err => { .catch(err => {
alert('Error updating class assignments or descriptions'); alert('Error updating class assignments or descriptions');
console.error(err); console.error(err);
}); });
}); });
}); });
}); });

544
js/start-training.js Normal file → Executable file
View File

@@ -1,272 +1,272 @@
// Render helper descriptions for YOLOX settings and handle form submission // Render helper descriptions for YOLOX settings and handle form submission
window.addEventListener('DOMContentLoaded', () => { window.addEventListener('DOMContentLoaded', () => {
// Get the form element at the top // Get the form element at the top
const form = document.getElementById('settings-form'); const form = document.getElementById('settings-form');
// Base config state // Base config state
let currentBaseConfig = null; let currentBaseConfig = null;
let baseConfigFields = []; let baseConfigFields = [];
// Define which fields are protected by base config // Define which fields are protected by base config
const protectedFields = [ const protectedFields = [
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr', 'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay', 'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup', 'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees', 'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval' 'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
]; ];
// Map backend field names to frontend field names // Map backend field names to frontend field names
const fieldNameMap = { const fieldNameMap = {
'activation': 'act', // Backend uses 'activation', frontend uses 'act' 'activation': 'act', // Backend uses 'activation', frontend uses 'act'
'nms_thre': 'nmsthre' 'nms_thre': 'nmsthre'
}; };
// Function to load base config for selected model // Function to load base config for selected model
function loadBaseConfig(modelName) { function loadBaseConfig(modelName) {
if (!modelName) return Promise.resolve(null); if (!modelName) return Promise.resolve(null);
return fetch(`/api/base-config/${modelName}`) return fetch(`/api/base-config/${modelName}`)
.then(res => { .then(res => {
if (!res.ok) throw new Error('Base config not found'); if (!res.ok) throw new Error('Base config not found');
return res.json(); return res.json();
}) })
.catch(err => { .catch(err => {
console.warn(`Could not load base config for ${modelName}:`, err); console.warn(`Could not load base config for ${modelName}:`, err);
return null; return null;
}); });
} }
// Function to apply base config to form fields // Function to apply base config to form fields
function applyBaseConfig(config, isCocoMode) { function applyBaseConfig(config, isCocoMode) {
const infoBanner = document.getElementById('base-config-info'); const infoBanner = document.getElementById('base-config-info');
const modelNameSpan = document.getElementById('base-config-model'); const modelNameSpan = document.getElementById('base-config-model');
if (!config || !isCocoMode) { if (!config || !isCocoMode) {
// Hide info banner // Hide info banner
if (infoBanner) infoBanner.style.display = 'none'; if (infoBanner) infoBanner.style.display = 'none';
// Remove grey styling and enable all fields // Remove grey styling and enable all fields
protectedFields.forEach(fieldName => { protectedFields.forEach(fieldName => {
const input = form.querySelector(`[name="${fieldName}"]`); const input = form.querySelector(`[name="${fieldName}"]`);
if (input) { if (input) {
input.disabled = false; input.disabled = false;
input.style.backgroundColor = '#f8f8f8'; input.style.backgroundColor = '#f8f8f8';
input.style.color = '#333'; input.style.color = '#333';
input.style.cursor = 'text'; input.style.cursor = 'text';
input.title = ''; input.title = '';
} }
}); });
baseConfigFields = []; baseConfigFields = [];
return; return;
} }
// Show info banner // Show info banner
if (infoBanner) { if (infoBanner) {
infoBanner.style.display = 'block'; infoBanner.style.display = 'block';
const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model'; const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model';
if (modelNameSpan) modelNameSpan.textContent = modelName; if (modelNameSpan) modelNameSpan.textContent = modelName;
} }
// Apply base config values and grey out fields // Apply base config values and grey out fields
baseConfigFields = []; baseConfigFields = [];
Object.entries(config).forEach(([key, value]) => { Object.entries(config).forEach(([key, value]) => {
// Map backend field name to frontend field name if needed // Map backend field name to frontend field name if needed
const frontendFieldName = fieldNameMap[key] || key; const frontendFieldName = fieldNameMap[key] || key;
if (protectedFields.includes(frontendFieldName)) { if (protectedFields.includes(frontendFieldName)) {
const input = form.querySelector(`[name="${frontendFieldName}"]`); const input = form.querySelector(`[name="${frontendFieldName}"]`);
if (input) { if (input) {
baseConfigFields.push(frontendFieldName); baseConfigFields.push(frontendFieldName);
// Set value based on type // Set value based on type
if (input.type === 'checkbox') { if (input.type === 'checkbox') {
input.checked = Boolean(value); input.checked = Boolean(value);
} else if (Array.isArray(value)) { } else if (Array.isArray(value)) {
input.value = value.join(','); input.value = value.join(',');
} else { } else {
input.value = value; input.value = value;
} }
// Grey out and disable // Grey out and disable
input.disabled = true; input.disabled = true;
input.style.backgroundColor = '#d3d3d3'; input.style.backgroundColor = '#d3d3d3';
input.style.color = '#666'; input.style.color = '#666';
input.style.cursor = 'not-allowed'; input.style.cursor = 'not-allowed';
// Add title tooltip // Add title tooltip
const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model'; const modelName = form.querySelector('[name="select_model"]')?.value || 'selected model';
input.title = `Protected by base config for ${modelName}. Switch to "Train from sketch" to customize.`; input.title = `Protected by base config for ${modelName}. Switch to "Train from sketch" to customize.`;
} }
} }
}); });
console.log(`Applied base config. Protected fields: ${baseConfigFields.join(', ')}`); console.log(`Applied base config. Protected fields: ${baseConfigFields.join(', ')}`);
} }
// Function to update form based on transfer learning mode // Function to update form based on transfer learning mode
function updateTransferLearningMode() { function updateTransferLearningMode() {
const transferLearning = document.getElementById('transfer-learning'); const transferLearning = document.getElementById('transfer-learning');
const selectModel = document.getElementById('select-model'); const selectModel = document.getElementById('select-model');
const selectModelRow = document.getElementById('select-model-row'); const selectModelRow = document.getElementById('select-model-row');
if (!transferLearning || !selectModel) return; if (!transferLearning || !selectModel) return;
const isCocoMode = transferLearning.value === 'coco'; const isCocoMode = transferLearning.value === 'coco';
const isCustomMode = transferLearning.value === 'custom'; const isCustomMode = transferLearning.value === 'custom';
const isSketchMode = transferLearning.value === 'sketch'; const isSketchMode = transferLearning.value === 'sketch';
const modelName = selectModel.value; const modelName = selectModel.value;
// Show/hide select model based on transfer learning mode // Show/hide select model based on transfer learning mode
if (selectModelRow) { if (selectModelRow) {
if (isSketchMode) { if (isSketchMode) {
selectModelRow.style.display = 'none'; selectModelRow.style.display = 'none';
} else { } else {
selectModelRow.style.display = ''; selectModelRow.style.display = '';
} }
} }
if (isCocoMode && modelName) { if (isCocoMode && modelName) {
// Load and apply base config // Load and apply base config
loadBaseConfig(modelName).then(config => { loadBaseConfig(modelName).then(config => {
currentBaseConfig = config; currentBaseConfig = config;
applyBaseConfig(config, true); applyBaseConfig(config, true);
}); });
} else { } else {
// Clear base config // Clear base config
currentBaseConfig = null; currentBaseConfig = null;
applyBaseConfig(null, false); applyBaseConfig(null, false);
} }
} }
// Listen for changes to transfer learning dropdown // Listen for changes to transfer learning dropdown
const transferLearningSelect = document.getElementById('transfer-learning'); const transferLearningSelect = document.getElementById('transfer-learning');
if (transferLearningSelect) { if (transferLearningSelect) {
transferLearningSelect.addEventListener('change', updateTransferLearningMode); transferLearningSelect.addEventListener('change', updateTransferLearningMode);
} }
// Listen for changes to model selection // Listen for changes to model selection
const modelSelect = document.getElementById('select-model'); const modelSelect = document.getElementById('select-model');
if (modelSelect) { if (modelSelect) {
modelSelect.addEventListener('change', updateTransferLearningMode); modelSelect.addEventListener('change', updateTransferLearningMode);
} }
// Initial update on page load // Initial update on page load
setTimeout(updateTransferLearningMode, 100); setTimeout(updateTransferLearningMode, 100);
// Auto-set num_classes from training_project classes array // Auto-set num_classes from training_project classes array
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id'); const projectId = urlParams.get('id');
if (projectId && form) { if (projectId && form) {
fetch('/api/training-projects') fetch('/api/training-projects')
.then(res => res.json()) .then(res => res.json())
.then(projects => { .then(projects => {
const project = projects.find(p => p.project_id == projectId || p.id == projectId); const project = projects.find(p => p.project_id == projectId || p.id == projectId);
if (project && project.classes) { if (project && project.classes) {
let classesArr = project.classes; let classesArr = project.classes;
// If classes is a stringified JSON, parse it // If classes is a stringified JSON, parse it
if (typeof classesArr === 'string') { if (typeof classesArr === 'string') {
try { try {
classesArr = JSON.parse(classesArr); classesArr = JSON.parse(classesArr);
} catch (e) { } catch (e) {
classesArr = []; classesArr = [];
} }
} }
let numClasses = 0; let numClasses = 0;
if (Array.isArray(classesArr)) { if (Array.isArray(classesArr)) {
numClasses = classesArr.length; numClasses = classesArr.length;
} else if (typeof classesArr === 'object' && classesArr !== null) { } else if (typeof classesArr === 'object' && classesArr !== null) {
numClasses = Object.keys(classesArr).length; numClasses = Object.keys(classesArr).length;
} }
// Fix: Only set num_classes if input exists // Fix: Only set num_classes if input exists
const numClassesInput = form.querySelector('[name="num_classes"]'); const numClassesInput = form.querySelector('[name="num_classes"]');
if (numClassesInput) { if (numClassesInput) {
numClassesInput.value = numClasses; numClassesInput.value = numClasses;
numClassesInput.readOnly = true; numClassesInput.readOnly = true;
numClassesInput.dispatchEvent(new Event('input')); numClassesInput.dispatchEvent(new Event('input'));
} }
} }
}); });
} }
// Handle form submission // Handle form submission
form.addEventListener('submit', function(e) { form.addEventListener('submit', function(e) {
console.log("Form submitted"); console.log("Form submitted");
e.preventDefault(); e.preventDefault();
// Temporarily enable disabled fields so they get included in FormData // Temporarily enable disabled fields so they get included in FormData
const disabledInputs = []; const disabledInputs = [];
form.querySelectorAll('input[disabled], select[disabled]').forEach(input => { form.querySelectorAll('input[disabled], select[disabled]').forEach(input => {
input.disabled = false; input.disabled = false;
disabledInputs.push(input); disabledInputs.push(input);
}); });
const formData = new FormData(form); const formData = new FormData(form);
const settings = {}; const settings = {};
let fileToUpload = null; let fileToUpload = null;
for (const [key, value] of formData.entries()) { for (const [key, value] of formData.entries()) {
if (key === 'model_upload' && form.elements[key].files.length > 0) { if (key === 'model_upload' && form.elements[key].files.length > 0) {
fileToUpload = form.elements[key].files[0]; fileToUpload = form.elements[key].files[0];
continue; continue;
} }
if (key === 'ema' || key === 'enable_mixup' || key === 'save_history_ckpt') { if (key === 'ema' || key === 'enable_mixup' || key === 'save_history_ckpt') {
settings[key] = form.elements[key].checked; settings[key] = form.elements[key].checked;
} else if (key === 'scale' || key === 'mosaic_scale' || key === 'mixup_scale' || key === 'input_size' || key === 'test_size') { } else if (key === 'scale' || key === 'mosaic_scale' || key === 'mixup_scale' || key === 'input_size' || key === 'test_size') {
settings[key] = value.split(',').map(v => parseFloat(v.trim())); settings[key] = value.split(',').map(v => parseFloat(v.trim()));
} else if (!isNaN(value) && value !== '') { } else if (!isNaN(value) && value !== '') {
settings[key] = parseFloat(value); settings[key] = parseFloat(value);
} else { } else {
settings[key] = value; settings[key] = value;
} }
} }
// Re-disable the inputs // Re-disable the inputs
disabledInputs.forEach(input => { disabledInputs.forEach(input => {
input.disabled = true; input.disabled = true;
}); });
// Attach project id from URL // Attach project id from URL
const urlParams = new URLSearchParams(window.location.search); const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id'); const projectId = urlParams.get('id');
if (projectId) settings.project_id = Number(projectId); if (projectId) settings.project_id = Number(projectId);
// First, send settings JSON (without file) // First, send settings JSON (without file)
fetch('/api/yolox-settings', { fetch('/api/yolox-settings', {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/json' }, headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(settings) body: JSON.stringify(settings)
}) })
.then(res => res.json()) .then(res => res.json())
.then(data => { .then(data => {
// If file selected, send it as binary // If file selected, send it as binary
if (fileToUpload) { if (fileToUpload) {
const reader = new FileReader(); const reader = new FileReader();
reader.onload = function(e) { reader.onload = function(e) {
fetch(`/api/yolox-settings/upload?project_id=${settings.project_id}`, { fetch(`/api/yolox-settings/upload?project_id=${settings.project_id}`, {
method: 'POST', method: 'POST',
headers: { 'Content-Type': 'application/octet-stream' }, headers: { 'Content-Type': 'application/octet-stream' },
body: e.target.result body: e.target.result
}) })
.then(res => res.json()) .then(res => res.json())
.then(data2 => { .then(data2 => {
alert('YOLOX settings and model file saved!'); alert('YOLOX settings and model file saved!');
window.location.href = `/overview-training.html?id=${settings.project_id}`; window.location.href = `/overview-training.html?id=${settings.project_id}`;
}) })
.catch(err => { .catch(err => {
alert('Error uploading model file'); alert('Error uploading model file');
console.error(err); console.error(err);
}); });
}; };
reader.readAsArrayBuffer(fileToUpload); reader.readAsArrayBuffer(fileToUpload);
} else { } else {
alert('YOLOX settings saved!'); alert('YOLOX settings saved!');
window.location.href = `/overview-training.html?id=${settings.project_id}`; window.location.href = `/overview-training.html?id=${settings.project_id}`;
} }
}) })
.catch(err => { .catch(err => {
alert('Error saving YOLOX settings'); alert('Error saving YOLOX settings');
console.error(err); console.error(err);
}); });
}); });
}); });

20
js/storage.js Normal file → Executable file
View File

@@ -1,11 +1,11 @@
// js/storage.js // js/storage.js
export function getStoredProjects() { export function getStoredProjects() {
try { try {
return JSON.parse(localStorage.getItem('ls_projects') || '{}'); return JSON.parse(localStorage.getItem('ls_projects') || '{}');
} catch (e) { } catch (e) {
return {}; return {};
} }
} }
export function setStoredProjects(projectsObj) { export function setStoredProjects(projectsObj) {
localStorage.setItem('ls_projects', JSON.stringify(projectsObj)); localStorage.setItem('ls_projects', JSON.stringify(projectsObj));
} }

0
media/logo.png Normal file → Executable file
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

709
overview-training.html Normal file → Executable file
View File

@@ -1,254 +1,457 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" <label id="project-title-label"
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label> style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <!-- Training Notification Bell -->
Training Project</button> <button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
<button id="seed-db-btn" class="button"> style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
Seed Database 🔔
<div class="loader" id="loader" style="display: none"></div> <span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
</div> text-align: center; font-weight: bold;">0</span>
</button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
</div> Training Project</button>
</button> <button id="seed-db-btn" class="button">
<button id="generate-yolox-json-btn" class="button"> Seed Database
Generate YOLOX JSON <div class="loader" id="loader" style="display: none"></div>
</button>
</button>
</button> <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
<button id="setup-details" class="button"> </div>
Show Details
</button>
</div>
<script> </button>
document.getElementById('seed-db-btn').addEventListener('click', function () {
const elLoader = document.getElementById("loader")
elLoader.style.display = "inherit"
<script>
fetch('/api/seed') document.getElementById('seed-db-btn').addEventListener('click', function () {
.finally(() => { const elLoader = document.getElementById("loader")
// Instead of hiding loader immediately, poll /api/update-status until done elLoader.style.display = "inherit"
function pollStatus() {
fetch('/api/update-status') fetch('/api/seed')
.then(res => res.json()) .finally(() => {
.then(status => { // Instead of hiding loader immediately, poll /api/update-status until done
if (status && status.running) { function pollStatus() {
// Still running, poll again after short delay fetch('/api/update-status')
setTimeout(pollStatus, 5000); .then(res => res.json())
} else { .then(status => {
elLoader.style.display = "none"; if (status && status.running) {
} // Still running, poll again after short delay
}) setTimeout(pollStatus, 5000);
.catch(() => { } else {
elLoader.style.display = "none"; elLoader.style.display = "none";
}); }
} })
pollStatus(); .catch(() => {
}) elLoader.style.display = "none";
}); });
}
// Show loader if backend is still processing on page load pollStatus();
})
function pollStatus() { });
const elLoader = document.getElementById("loader");
fetch('/api/update-status') // Show loader if backend is still processing on page load
.then(res => res.json())
function pollStatus() {
.then(status => { const elLoader = document.getElementById("loader");
if (status && status.running) { fetch('/api/update-status')
elLoader.style.display = "inherit"; .then(res => res.json())
setTimeout(pollStatus, 5000);
} else { .then(status => {
elLoader.style.display = "none"; if (status && status.running) {
} elLoader.style.display = "inherit";
}) setTimeout(pollStatus, 5000);
.catch(() => { } else {
elLoader.style.display = "none"; elLoader.style.display = "none";
}); }
} })
.catch(() => {
</script> elLoader.style.display = "none";
<script> });
// Declare urlParams and projectId once }
const urlParams = new URLSearchParams(window.location.search);
const projectId = urlParams.get('id'); </script>
// Set project title in header <script>
fetch('/api/training-projects') // Declare urlParams and projectId once
.then(res => res.json()) const urlParams = new URLSearchParams(window.location.search);
.then(projects => { const projectId = urlParams.get('id');
const project = projects.find(p => p.project_id == projectId || p.id == projectId); // Set project title in header
if (project) { fetch('/api/training-projects')
const titleLabel = document.getElementById('project-title-label'); .then(res => res.json())
if (titleLabel) titleLabel.textContent = '/' + project.title; .then(projects => {
} const project = projects.find(p => p.project_id == projectId || p.id == projectId);
}); if (project) {
// Render trainings const titleLabel = document.getElementById('project-title-label');
function renderTrainings(trainings) { if (titleLabel) titleLabel.textContent = '/' + project.title;
const list = document.getElementById('projects-list'); }
list.innerHTML = ''; });
if (!Array.isArray(trainings) || trainings.length === 0) { // Render trainings
list.innerHTML = '<div style="color:#009eac;padding:16px;">No trainings found for this project.</div>'; function renderTrainings(trainings) {
return; const list = document.getElementById('projects-list');
} list.innerHTML = '';
trainings.forEach(training => { if (!Array.isArray(trainings) || trainings.length === 0) {
const card = document.createElement('div'); list.innerHTML = '<div style="color:#009eac;padding:16px;">No trainings found for this project.</div>';
card.className = 'dataset-card'; return;
card.style = 'border:1px solid #009eac;padding:12px;margin:8px;border-radius:8px;background:#eaf7fa;position:relative;min-width:320px;min-height:120px;display:flex;flex-direction:row;justify-content:space-between;align-items:stretch;'; }
trainings.forEach(training => {
// Info section (left) const card = document.createElement('div');
const infoDiv = document.createElement('div'); card.className = 'dataset-card';
infoDiv.style = 'flex:1; text-align:left;'; card.style = 'border:1px solid #009eac;padding:12px;margin:8px;border-radius:8px;background:#eaf7fa;position:relative;min-width:320px;min-height:120px;display:flex;flex-direction:row;justify-content:space-between;align-items:stretch;';
infoDiv.innerHTML = `<b>${training.exp_name || 'Training'}</b><br>Epochs: ${training.max_epoch}<br>Depth: ${training.depth}<br>Width: ${training.width}<br>Activation: ${training.activation || training.act || ''}`;
// Info section (left)
// Buttons section (right) const infoDiv = document.createElement('div');
const btnDiv = document.createElement('div'); infoDiv.style = 'flex:1; text-align:left;';
btnDiv.style = 'display:flex;flex-direction:column;align-items:flex-end;gap:8px;min-width:160px;'; infoDiv.innerHTML = `<b>${training.exp_name || 'Training'}</b><br>Epochs: ${training.max_epoch}<br>Depth: ${training.depth}<br>Width: ${training.width}<br>Activation: ${training.activation || training.act || ''}`;
// Start Training button // Buttons section (right)
const startBtn = document.createElement('button'); const btnDiv = document.createElement('div');
startBtn.textContent = 'Start YOLOX Training'; btnDiv.style = 'display:flex;flex-direction:column;align-items:flex-end;gap:8px;min-width:160px;';
startBtn.style = 'background:#009eac;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
startBtn.onclick = function() { // Start Training button
startBtn.disabled = true; const startBtn = document.createElement('button');
fetch('/api/start-yolox-training', { startBtn.textContent = 'Start YOLOX Training';
method: 'POST', startBtn.style = 'background:#009eac;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
headers: { 'Content-Type': 'application/json' }, startBtn.onclick = function() {
body: JSON.stringify({ project_id: projectId, training_id: training.id }) startBtn.disabled = true;
}) fetch('/api/start-yolox-training', {
.then(res => res.json()) method: 'POST',
.then(result => { headers: { 'Content-Type': 'application/json' },
alert(result.message || 'Training started'); body: JSON.stringify({ project_id: projectId, training_id: training.id })
startBtn.disabled = false; })
}) .then(res => res.json())
.catch(() => { .then(result => {
alert('Failed to start training'); alert(result.message || 'Training started');
startBtn.disabled = false; startBtn.disabled = false;
}); })
}; .catch(() => {
btnDiv.appendChild(startBtn); alert('Failed to start training');
startBtn.disabled = false;
// View Log button });
const logBtn = document.createElement('button'); };
logBtn.textContent = 'View Training Log'; btnDiv.appendChild(startBtn);
logBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
logBtn.onclick = function() { // View Training Details button
showLogModal(training.id); const detailsBtn = document.createElement('button');
}; detailsBtn.textContent = 'View Training Details';
btnDiv.appendChild(logBtn); detailsBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
detailsBtn.onclick = function() {
// Remove button // Navigate to edit-training page in read-only mode
const removeBtn = document.createElement('button'); window.location.href = `/edit-training.html?training_id=${training.id}&readonly=true`;
removeBtn.textContent = 'Remove'; };
removeBtn.style = 'background:#ff4d4f;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;'; btnDiv.appendChild(detailsBtn);
removeBtn.onclick = function() {
if (confirm('Are you sure you want to delete this training?')) { // Remove button
fetch(`/api/trainings/${training.id}`, { method: 'DELETE' }) const removeBtn = document.createElement('button');
.then(res => res.json()) removeBtn.textContent = 'Remove';
.then(result => { removeBtn.style = 'background:#ff4d4f;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
alert(result.message || 'Training deleted'); removeBtn.onclick = function() {
fetchTrainings(); // Refresh list if (confirm('Are you sure you want to delete this training?')) {
}) fetch(`/api/trainings/${training.id}`, { method: 'DELETE' })
.catch(() => alert('Failed to delete training')); .then(res => res.json())
} .then(result => {
}; alert(result.message || 'Training deleted');
btnDiv.appendChild(removeBtn); fetchTrainings(); // Refresh list
})
card.appendChild(infoDiv); .catch(() => alert('Failed to delete training'));
card.appendChild(btnDiv); }
list.appendChild(card); };
}); btnDiv.appendChild(removeBtn);
// Modal for log display
if (!document.getElementById('log-modal')) { card.appendChild(infoDiv);
const modal = document.createElement('div'); card.appendChild(btnDiv);
modal.id = 'log-modal'; list.appendChild(card);
modal.style = 'display:none;position:fixed;top:0;left:0;width:100vw;height:100vh;background:rgba(0,0,0,0.5);z-index:9999;justify-content:center;align-items:center;'; });
modal.innerHTML = `<div style="background:#fff;padding:24px;border-radius:8px;max-width:800px;width:90vw;max-height:80vh;overflow:auto;position:relative;"><pre id='log-content' style='font-size:13px;white-space:pre-wrap;word-break:break-all;max-height:60vh;overflow:auto;background:#f7f7f7;padding:12px;border-radius:6px;'></pre><button id='close-log-modal' style='position:absolute;top:8px;right:8px;background:#009eac;color:#fff;border:none;border-radius:4px;padding:6px 12px;cursor:pointer;'>Close</button></div>`; // Modal for log display
document.body.appendChild(modal); if (!document.getElementById('log-modal')) {
document.getElementById('close-log-modal').onclick = function() { const modal = document.createElement('div');
modal.style.display = 'none'; modal.id = 'log-modal';
}; modal.style = 'display:none;position:fixed;top:0;left:0;width:100vw;height:100vh;background:rgba(0,0,0,0.5);z-index:9999;justify-content:center;align-items:center;';
} modal.innerHTML = `<div style="background:#fff;padding:24px;border-radius:8px;max-width:800px;width:90vw;max-height:80vh;overflow:auto;position:relative;"><pre id='log-content' style='font-size:13px;white-space:pre-wrap;word-break:break-all;max-height:60vh;overflow:auto;background:#f7f7f7;padding:12px;border-radius:6px;'></pre><button id='close-log-modal' style='position:absolute;top:8px;right:8px;background:#009eac;color:#fff;border:none;border-radius:4px;padding:6px 12px;cursor:pointer;'>Close</button></div>`;
document.body.appendChild(modal);
// Function to show log modal and poll log document.getElementById('close-log-modal').onclick = function() {
function showLogModal(trainingId) { modal.style.display = 'none';
const modal = document.getElementById('log-modal'); };
const logContent = document.getElementById('log-content'); }
modal.style.display = 'flex';
function fetchLog() { // Function to show log modal and poll log
fetch(`/api/training-log?project_id=${projectId}&training_id=${trainingId}`) function showLogModal(trainingId) {
.then(res => res.json()) const modal = document.getElementById('log-modal');
.then(data => { const logContent = document.getElementById('log-content');
logContent.textContent = data.log || 'No log found.'; modal.style.display = 'flex';
}) function fetchLog() {
.catch(() => { fetch(`/api/training-log?project_id=${projectId}&training_id=${trainingId}`)
logContent.textContent = 'Failed to fetch log.'; .then(res => res.json())
}); .then(data => {
} logContent.textContent = data.log || 'No log found.';
fetchLog(); })
// Poll every 5 seconds while modal is open .catch(() => {
let poller = setInterval(() => { logContent.textContent = 'Failed to fetch log.';
if (modal.style.display === 'flex') fetchLog(); });
else clearInterval(poller); }
}, 5000); fetchLog();
} // Poll every 5 seconds while modal is open
} let poller = setInterval(() => {
// Fetch trainings for project if (modal.style.display === 'flex') fetchLog();
function fetchTrainings() { else clearInterval(poller);
if (!projectId) return; }, 5000);
fetch(`/api/trainings?project_id=${projectId}`) }
.then(res => res.json()) }
.then(trainings => { // Fetch trainings for project
renderTrainings(trainings); function fetchTrainings() {
}); if (!projectId) return;
} fetch(`/api/trainings?project_id=${projectId}`)
window.addEventListener('DOMContentLoaded', fetchTrainings); .then(res => res.json())
document.getElementById('generate-yolox-json-btn').addEventListener('click', function () { .then(trainings => {
fetch('/api/generate-yolox-json', { renderTrainings(trainings);
method: 'POST', });
headers: { 'Content-Type': 'application/json' }, }
body: JSON.stringify({ project_id: projectId }) window.addEventListener('DOMContentLoaded', fetchTrainings);
}) </script>
.then(res => res.json()) <div style="padding: 16px; text-align: left;">
.then(result => { <button id="create-new-training-btn" class="button" style="background:#009eac;color:white;">
alert('YOLOX JSON generated!'); + Create New Training
}) </button>
.catch(err => { </div>
alert('Failed to generate YOLOX JSON'); <div id="projects-list"></div>
}); </div>
});
</script> <script>
<div id="projects-list"></div> // Create New Training button handler
</div> document.addEventListener('DOMContentLoaded', function() {
</body> document.getElementById('create-new-training-btn').addEventListener('click', function() {
if (!projectId) {
alert('No project selected');
return;
}
// Navigate to edit-training page to configure new training parameters
// This will reuse existing project details and class mappings
window.location.href = `/edit-training.html?project_id=${projectId}`;
});
});
</script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus(); // Immediate update
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
// Update bell appearance
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
// Update modal content
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
// Poll every 5 seconds
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
// Stop polling when page unloads
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html> </html>

421
project-details.html Normal file → Executable file
View File

@@ -1,116 +1,307 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon> style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label> <label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" <!-- Training Notification Bell -->
class="button-red">Add Training Project</button> <button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
<button id="seed-db-btn" class="button"> style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
Seed Database 🔔
<div class="loader" id="loader" style="display: none"></div> <span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
</button> text-align: center; font-weight: bold;">0</span>
</div> </button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'"
</div> class="button-red">Add Training Project</button>
<button id="seed-db-btn" class="button">
<div id="projects-list"> Seed Database
<div class="loader" id="loader" style="display: none"></div>
</div>
<script> </button>
document.getElementById('seed-db-btn').addEventListener('click', function () { <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
const elLoader = document.getElementById("loader") </div>
elLoader.style.display = "inherit"
</div>
fetch('/api/seed')
.finally(() => { <div id="projects-list">
// Instead of hiding loader immediately, poll /api/update-status until done
function pollStatus() { </div>
fetch('/api/update-status') <script>
.then(res => res.json()) document.getElementById('seed-db-btn').addEventListener('click', function () {
.then(status => { const elLoader = document.getElementById("loader")
if (status && status.running) { elLoader.style.display = "inherit"
// Still running, poll again after short delay
setTimeout(pollStatus, 5000); fetch('/api/seed')
} else { .finally(() => {
elLoader.style.display = "none"; // Instead of hiding loader immediately, poll /api/update-status until done
} function pollStatus() {
}) fetch('/api/update-status')
.catch(() => { .then(res => res.json())
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} // Still running, poll again after short delay
pollStatus(); setTimeout(pollStatus, 5000);
}) } else {
}); elLoader.style.display = "none";
}
// Show loader if backend is still processing on page load })
.catch(() => {
function pollStatus() { elLoader.style.display = "none";
const elLoader = document.getElementById("loader"); });
fetch('/api/update-status') }
.then(res => res.json()) pollStatus();
})
.then(status => { });
if (status && status.running) {
elLoader.style.display = "inherit"; // Show loader if backend is still processing on page load
setTimeout(pollStatus, 5000);
} else { function pollStatus() {
elLoader.style.display = "none"; const elLoader = document.getElementById("loader");
} fetch('/api/update-status')
}) .then(res => res.json())
.catch(() => {
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} elLoader.style.display = "inherit";
setTimeout(pollStatus, 5000);
</script> } else {
<script> elLoader.style.display = "none";
const urlParams = new URLSearchParams(window.location.search); }
const projectId = urlParams.get('id'); })
fetch('/api/training-projects') .catch(() => {
.then(res => res.json()) elLoader.style.display = "none";
.then(projects => { });
const project = projects.find(p => p.project_id == projectId || p.id == projectId); }
if (project) {
document.getElementById('project-title-label').textContent = '/' + project.title; </script>
} <script>
}); const urlParams = new URLSearchParams(window.location.search);
</script> const projectId = urlParams.get('id');
fetch('/api/training-projects')
.then(res => res.json())
.then(projects => {
</div> const project = projects.find(p => p.project_id == projectId || p.id == projectId);
<script src="./js/dashboard-label-studio.js"></script> if (project) {
</body> document.getElementById('project-title-label').textContent = '/' + project.title;
}
});
</script>
</div>
<script src="./js/dashboard-label-studio.js"></script>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
</div>
</div>
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
</div>
</div>
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html> </html>

236
settings.html Executable file
View File

@@ -0,0 +1,236 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>Settings - mb ai Trainer</title>
<link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" />
<style>
.modal {
display: none;
position: fixed;
z-index: 1000;
left: 0;
top: 0;
width: 100%;
height: 100%;
overflow: auto;
background-color: rgba(0,0,0,0.4);
}
.modal-content {
background-color: #fefefe;
margin: 5% auto;
padding: 20px;
border: 1px solid #888;
border-radius: 8px;
width: 600px;
max-width: 90%;
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 20px;
border-bottom: 2px solid #e0e0e0;
padding-bottom: 10px;
}
.modal-header h2 {
margin: 0;
color: #333;
}
.close {
color: #aaa;
font-size: 28px;
font-weight: bold;
cursor: pointer;
}
.close:hover,
.close:focus {
color: #000;
}
.settings-section {
margin-bottom: 25px;
}
.settings-section h3 {
margin-bottom: 10px;
color: #555;
font-size: 18px;
}
.setting-row {
margin-bottom: 15px;
}
.setting-row label {
display: block;
margin-bottom: 5px;
font-weight: 600;
color: #333;
}
.setting-row input {
width: 100%;
padding: 8px;
border: 1px solid #ddd;
border-radius: 4px;
box-sizing: border-box;
}
.setting-row small {
display: block;
margin-top: 4px;
color: #666;
font-size: 12px;
}
.button-row {
display: flex;
gap: 10px;
margin-top: 10px;
}
.test-button {
padding: 6px 12px;
background-color: #4CAF50;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 14px;
}
.test-button:hover {
background-color: #45a049;
}
.test-button:disabled {
background-color: #cccccc;
cursor: not-allowed;
}
.save-button {
padding: 10px 20px;
background-color: #2196F3;
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 16px;
}
.save-button:hover {
background-color: #0b7dda;
}
.status-message {
padding: 10px;
margin-top: 10px;
border-radius: 4px;
font-size: 14px;
display: none;
}
.status-success {
background-color: #d4edda;
color: #155724;
border: 1px solid #c3e6cb;
}
.status-error {
background-color: #f8d7da;
color: #721c24;
border: 1px solid #f5c6cb;
}
.loader-small {
border: 3px solid #f3f3f3;
border-top: 3px solid #3498db;
border-radius: 50%;
width: 16px;
height: 16px;
animation: spin 1s linear infinite;
display: inline-block;
margin-left: 8px;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
</style>
</head>
<body>
<!-- Settings Modal -->
<div id="settingsModal" class="modal">
<div class="modal-content">
<div class="modal-header">
<h2>⚙️ Global Settings</h2>
<span class="close">&times;</span>
</div>
<div class="settings-section">
<h3>Label Studio Connection</h3>
<div class="setting-row">
<label for="labelstudio_api_url">API URL:</label>
<input type="text" id="labelstudio_api_url" placeholder="http://192.168.1.19:8080/api" />
<small>Enter the base URL of your Label Studio API</small>
</div>
<div class="setting-row">
<label for="labelstudio_api_token">API Token:</label>
<input type="text" id="labelstudio_api_token" placeholder="Your API token" />
<small>Find your API token in Label Studio Account settings</small>
</div>
<div class="button-row">
<button class="test-button" id="testLabelStudioBtn">Test Connection</button>
</div>
<div id="labelstudioTestResult" class="status-message"></div>
</div>
<div class="settings-section">
<h3>YOLOX Installation</h3>
<div class="setting-row">
<label for="yolox_path">YOLOX Path:</label>
<input type="text" id="yolox_path" placeholder="C:/YOLOX" />
<small>Enter the path to your YOLOX installation directory</small>
</div>
<div class="setting-row">
<label for="yolox_venv_path">YOLOX Virtual Environment Path:</label>
<input type="text" id="yolox_venv_path" placeholder="e.g., /path/to/yolox_venv" />
<small>Path to YOLOX venv folder or activation script</small>
</div>
<div class="setting-row">
<label for="yolox_output_path">YOLOX Output Folder:</label>
<input type="text" id="yolox_output_path" placeholder="./backend" />
<small>Folder where experiment files (exp.py, exp_infer.py) and JSON files will be saved</small>
</div>
<div class="button-row">
<button class="test-button" id="testYoloxBtn">Verify Path</button>
</div>
<div id="yoloxTestResult" class="status-message"></div>
</div>
<div style="text-align: right; margin-top: 20px; padding-top: 20px; border-top: 1px solid #e0e0e0;">
<button class="save-button" id="saveSettingsBtn">Save Settings</button>
</div>
<div id="saveResult" class="status-message"></div>
</div>
</div>
<script src="js/settings.js"></script>
</body>
</html>

413
setup-training-project.html Normal file → Executable file
View File

@@ -1,109 +1,306 @@
<!DOCTYPE html> <!DOCTYPE html>
<html> <html>
<head> <head>
<meta name="viewport" content="width=device-width, initial-scale=1" /> <meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" /> <meta charset="utf-8" />
<link rel="stylesheet" href="globals.css" /> <link rel="stylesheet" href="globals.css" />
<link rel="stylesheet" href="styleguide.css" /> <link rel="stylesheet" href="styleguide.css" />
<link rel="stylesheet" href="style.css" /> <link rel="stylesheet" href="style.css" />
<style> <style>
#projects-list { #projects-list {
display: flex; display: flex;
flex-wrap: wrap; flex-wrap: wrap;
gap: 15px; gap: 15px;
justify-content: flex-start; justify-content: flex-start;
align-items: flex-start; align-items: flex-start;
} }
.dataset-card { .dataset-card {
flex: 0 0 auto; flex: 0 0 auto;
} }
</style> </style>
</head> </head>
<body onload="pollStatus()"> <body onload="pollStatus()">
<div> <div>
<div id="header"> <div id="header">
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;" <icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
src="./media/logo.png" alt="Logo"></icon> src="./media/logo.png" alt="Logo"></icon>
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label> <label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
<div class="button-row"> <div class="button-row">
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add <!-- Training Notification Bell -->
Training Project</button> <button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
<button id="seed-db-btn" class="button"> style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
Seed Database 🔔
<div class="loader" id="loader" style="display: none"></div> <span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
</button> text-align: center; font-weight: bold;">0</span>
</div> </button>
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
</div> Training Project</button>
<div id="details"> <button id="seed-db-btn" class="button">
<script src="js/setup-training-project.js"></script> Seed Database
</div> <div class="loader" id="loader" style="display: none"></div>
<script>
document.getElementById('seed-db-btn').addEventListener('click', function () { </button>
const elLoader = document.getElementById("loader") <button id="settings-btn" onclick="window.openSettingsModal()" class="button" title="Einstellungen" style="padding: 8px 12px; margin-left: 10px;">⚙️</button>
elLoader.style.display = "inherit" </div>
fetch('/api/seed') </div>
.finally(() => { <div id="details">
// Instead of hiding loader immediately, poll /api/update-status until done <script src="js/setup-training-project.js"></script>
function pollStatus() { </div>
fetch('/api/update-status') <script>
.then(res => res.json()) document.getElementById('seed-db-btn').addEventListener('click', function () {
.then(status => { const elLoader = document.getElementById("loader")
if (status && status.running) { elLoader.style.display = "inherit"
// Still running, poll again after short delay
setTimeout(pollStatus, 5000); fetch('/api/seed')
} else { .finally(() => {
elLoader.style.display = "none"; // Instead of hiding loader immediately, poll /api/update-status until done
} function pollStatus() {
}) fetch('/api/update-status')
.catch(() => { .then(res => res.json())
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} // Still running, poll again after short delay
pollStatus(); setTimeout(pollStatus, 5000);
}) } else {
}); elLoader.style.display = "none";
}
// Show loader if backend is still processing on page load })
.catch(() => {
function pollStatus() { elLoader.style.display = "none";
const elLoader = document.getElementById("loader"); });
fetch('/api/update-status') }
.then(res => res.json()) pollStatus();
})
.then(status => { });
if (status && status.running) {
elLoader.style.display = "inherit"; // Show loader if backend is still processing on page load
setTimeout(pollStatus, 5000);
} else { function pollStatus() {
elLoader.style.display = "none"; const elLoader = document.getElementById("loader");
} fetch('/api/update-status')
}) .then(res => res.json())
.catch(() => {
elLoader.style.display = "none"; .then(status => {
}); if (status && status.running) {
} elLoader.style.display = "inherit";
setTimeout(pollStatus, 5000);
</script> } else {
<script> elLoader.style.display = "none";
const urlParams = new URLSearchParams(window.location.search); }
const projectId = urlParams.get('id'); })
fetch('/api/training-projects') .catch(() => {
.then(res => res.json()) elLoader.style.display = "none";
.then(projects => { });
const project = projects.find(p => p.project_id == projectId || p.id == projectId); }
if (project) {
document.getElementById('project-title-label').textContent = '/' + project.title; </script>
} <script>
}); const urlParams = new URLSearchParams(window.location.search);
</script> const projectId = urlParams.get('id');
fetch('/api/training-projects')
</div> .then(res => res.json())
</body> .then(projects => {
const project = projects.find(p => p.project_id == projectId || p.id == projectId);
if (project) {
document.getElementById('project-title-label').textContent = '/' + project.title;
}
});
</script>
</div>
<!-- Settings Modal -->
<div id="settings-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 600px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Einstellungen</h2>
<button class="close-btn" onclick="window.closeSettingsModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Label Studio Settings -->
<div class="settings-section">
<h3>Label Studio</h3>
<div class="form-group">
<label for="labelstudio-url">API URL:</label>
<input type="text" id="labelstudio-url" placeholder="http://192.168.1.19:8080">
</div>
<div class="form-group">
<label for="labelstudio-token">API Token:</label>
<input type="password" id="labelstudio-token" placeholder="API Token">
</div>
<button id="test-labelstudio-btn" class="button">
Verbindung testen
<div class="loader" id="test-ls-loader" style="display: none"></div>
</button>
<div id="labelstudio-status" class="status-message"></div>
</div>
<!-- YOLOX Settings -->
<div class="settings-section">
<h3>YOLOX</h3>
<div class="form-group">
<label for="yolox-path">Installation Path:</label>
<input type="text" id="yolox-path" placeholder="C:/YOLOX">
</div>
<div class="form-group">
<label for="yolox-venv-path">Virtual Environment Path:</label>
<input type="text" id="yolox-venv-path" placeholder="/path/to/venv/bin/activate">
</div>
<div class="form-group">
<label for="yolox-output-path">Output Folder:</label>
<input type="text" id="yolox-output-path" placeholder="./backend">
<small style="display: block; margin-top: 4px; color: #666;">Folder for experiment files and JSON files</small>
</div>
<div class="form-group">
<label for="yolox-data-dir">Data Directory:</label>
<input type="text" id="yolox-data-dir" placeholder="/home/kitraining/To_Annotate/">
<small style="display: block; margin-top: 4px; color: #666;">Path where training images are located</small>
</div>
<button id="test-yolox-btn" class="button">
Pfad überprüfen
<div class="loader" id="test-yolox-loader" style="display: none"></div>
</button>
<div id="yolox-status" class="status-message"></div>
</div>
<!-- Save Button -->
<div class="settings-section">
<button id="save-settings-btn" class="button-red">Einstellungen speichern</button>
<div id="save-status" class="status-message"></div>
</div>
</div>
</div>
</div>
<!-- Training Status Modal -->
<div id="training-status-modal" class="modal" style="display: none;">
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
<div class="modal-header">
<h2>Training Status</h2>
<button class="close-btn" onclick="toggleTrainingModal()">&times;</button>
</div>
<div class="modal-body">
<!-- Current Training -->
<div class="settings-section" id="current-training-section" style="display: none;">
<h3 style="color: #009eac;">Current Training</h3>
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
<!-- Populated by JS -->
</div>
</div>
<!-- Queued Trainings -->
<div class="settings-section" id="queue-section" style="display: none;">
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
<!-- Populated by JS -->
</div>
</div>
<!-- No trainings message -->
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
No trainings running or queued.
</div>
</div>
</div>
</div>
<script src="js/settings.js"></script>
<script>
// Training status polling
let trainingStatusPoller = null;
function toggleTrainingModal() {
const modal = document.getElementById('training-status-modal');
if (modal.style.display === 'none') {
modal.style.display = 'flex';
updateTrainingStatus();
} else {
modal.style.display = 'none';
}
}
function updateTrainingStatus() {
fetch('/api/training-status')
.then(res => res.json())
.then(data => {
const bell = document.getElementById('training-bell');
const badge = document.getElementById('bell-badge');
const currentSection = document.getElementById('current-training-section');
const queueSection = document.getElementById('queue-section');
const noTrainingsMsg = document.getElementById('no-trainings-msg');
const totalCount = (data.current ? 1 : 0) + data.queue.length;
if (totalCount > 0) {
bell.style.background = '#009eac';
badge.style.display = 'block';
badge.textContent = totalCount;
} else {
bell.style.background = '#999';
badge.style.display = 'none';
}
if (data.current) {
currentSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
document.getElementById('current-training-info').innerHTML = `
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
<strong>${data.current.name || 'Training'}</strong>
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
</div>
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
</div>
<div style="font-size: 14px; color: #666;">
Epoch ${data.current.iteration} / ${data.current.max_epoch}
</div>
`;
} else {
currentSection.style.display = 'none';
}
if (data.queue.length > 0) {
queueSection.style.display = 'block';
noTrainingsMsg.style.display = 'none';
document.getElementById('queue-count').textContent = data.queue.length;
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
<div style="font-size: 13px; color: #666; margin-top: 4px;">
${t.max_epoch} epochs • Waiting...
</div>
</div>
`).join('');
} else {
queueSection.style.display = 'none';
}
if (totalCount === 0) {
noTrainingsMsg.style.display = 'block';
}
})
.catch(err => console.error('Failed to fetch training status:', err));
}
window.addEventListener('DOMContentLoaded', function() {
updateTrainingStatus();
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
});
window.addEventListener('beforeunload', function() {
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
});
</script>
</body>
</html> </html>

0
start-training.html Normal file → Executable file
View File

1442
style.css Normal file → Executable file
View File

File diff suppressed because it is too large Load Diff

32
styleguide.css Normal file → Executable file
View File

@@ -1,16 +1,16 @@
:root { :root {
--m3-body-small-font-family: "Roboto", Helvetica; --m3-body-small-font-family: "Roboto", Helvetica;
--m3-body-small-font-weight: 400; --m3-body-small-font-weight: 400;
--m3-body-small-font-size: 12px; --m3-body-small-font-size: 12px;
--m3-body-small-letter-spacing: 0.4000000059604645px; --m3-body-small-letter-spacing: 0.4000000059604645px;
--m3-body-small-line-height: 16px; --m3-body-small-line-height: 16px;
--m3-body-small-font-style: normal; --m3-body-small-font-style: normal;
--title2-regular-font-family: "SF Pro", Helvetica; --title2-regular-font-family: "SF Pro", Helvetica;
--title2-regular-font-weight: 400; --title2-regular-font-weight: 400;
--title2-regular-font-size: 22px; --title2-regular-font-size: 22px;
--title2-regular-letter-spacing: -0.25999999046325684px; --title2-regular-letter-spacing: -0.25999999046325684px;
--title2-regular-line-height: 28px; --title2-regular-line-height: 28px;
--title2-regular-font-style: normal; --title2-regular-font-style: normal;
--minus-for-button-size: 30px; --minus-for-button-size: 30px;
} }

78
text.css Normal file → Executable file
View File

@@ -1,40 +1,40 @@
.popup .upload-button { .popup .upload-button {
width: 80px; width: 80px;
height: 25px; height: 25px;
background-color: #4cdb0085; background-color: #4cdb0085;
border-radius: 5px; border-radius: 5px;
display: flex; display: flex;
justify-content: center; justify-content: center;
align-items: center; align-items: center;
cursor: pointer; cursor: pointer;
position: absolute; position: absolute;
top: 303px; top: 303px;
left: 128px; left: 128px;
box-sizing: border-box; box-sizing: border-box;
} }
.popup .upload-button:hover{ .popup .upload-button:hover{
width: 80px; width: 80px;
height: 25px; height: 25px;
background-color: #36990085; background-color: #36990085;
border-radius: 5px; border-radius: 5px;
display: flex; display: flex;
justify-content: center; justify-content: center;
align-items: center; align-items: center;
cursor: pointer; cursor: pointer;
position: absolute; position: absolute;
top: 303px; top: 303px;
left: 128px; left: 128px;
box-sizing: border-box; box-sizing: border-box;
} }
.popup .upload-button-text { .popup .upload-button-text {
font-family: var(--m3-body-small-font-family); font-family: var(--m3-body-small-font-family);
font-weight: var(--m3-body-small-font-weight); font-weight: var(--m3-body-small-font-weight);
color: #000000; color: #000000;
font-size: var(--m3-body-small-font-size); font-size: var(--m3-body-small-font-size);
letter-spacing: var(--m3-body-small-letter-spacing); letter-spacing: var(--m3-body-small-letter-spacing);
line-height: var(--m3-body-small-line-height); line-height: var(--m3-body-small-line-height);
white-space: nowrap; white-space: nowrap;
font-style: var(--m3-body-small-font-style); font-style: var(--m3-body-small-font-style);
} }

247
venv/bin/Activate.ps1 Executable file
View File

@@ -0,0 +1,247 @@
<#
.Synopsis
Activate a Python virtual environment for the current PowerShell session.
.Description
Pushes the python executable for a virtual environment to the front of the
$Env:PATH environment variable and sets the prompt to signify that you are
in a Python virtual environment. Makes use of the command line switches as
well as the `pyvenv.cfg` file values present in the virtual environment.
.Parameter VenvDir
Path to the directory that contains the virtual environment to activate. The
default value for this is the parent of the directory that the Activate.ps1
script is located within.
.Parameter Prompt
The prompt prefix to display when this virtual environment is activated. By
default, this prompt is the name of the virtual environment folder (VenvDir)
surrounded by parentheses and followed by a single space (ie. '(.venv) ').
.Example
Activate.ps1
Activates the Python virtual environment that contains the Activate.ps1 script.
.Example
Activate.ps1 -Verbose
Activates the Python virtual environment that contains the Activate.ps1 script,
and shows extra information about the activation as it executes.
.Example
Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv
Activates the Python virtual environment located in the specified location.
.Example
Activate.ps1 -Prompt "MyPython"
Activates the Python virtual environment that contains the Activate.ps1 script,
and prefixes the current prompt with the specified string (surrounded in
parentheses) while the virtual environment is active.
.Notes
On Windows, it may be required to enable this Activate.ps1 script by setting the
execution policy for the user. You can do this by issuing the following PowerShell
command:
PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
For more information on Execution Policies:
https://go.microsoft.com/fwlink/?LinkID=135170
#>
Param(
[Parameter(Mandatory = $false)]
[String]
$VenvDir,
[Parameter(Mandatory = $false)]
[String]
$Prompt
)
<# Function declarations --------------------------------------------------- #>
<#
.Synopsis
Remove all shell session elements added by the Activate script, including the
addition of the virtual environment's Python executable from the beginning of
the PATH variable.
.Parameter NonDestructive
If present, do not remove this function from the global namespace for the
session.
#>
function global:deactivate ([switch]$NonDestructive) {
# Revert to original values
# The prior prompt:
if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) {
Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt
Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT
}
# The prior PYTHONHOME:
if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) {
Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME
Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME
}
# The prior PATH:
if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) {
Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH
Remove-Item -Path Env:_OLD_VIRTUAL_PATH
}
# Just remove the VIRTUAL_ENV altogether:
if (Test-Path -Path Env:VIRTUAL_ENV) {
Remove-Item -Path env:VIRTUAL_ENV
}
# Just remove VIRTUAL_ENV_PROMPT altogether.
if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) {
Remove-Item -Path env:VIRTUAL_ENV_PROMPT
}
# Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether:
if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) {
Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force
}
# Leave deactivate function in the global namespace if requested:
if (-not $NonDestructive) {
Remove-Item -Path function:deactivate
}
}
<#
.Description
Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the
given folder, and returns them in a map.
For each line in the pyvenv.cfg file, if that line can be parsed into exactly
two strings separated by `=` (with any amount of whitespace surrounding the =)
then it is considered a `key = value` line. The left hand string is the key,
the right hand is the value.
If the value starts with a `'` or a `"` then the first and last character is
stripped from the value before being captured.
.Parameter ConfigDir
Path to the directory that contains the `pyvenv.cfg` file.
#>
function Get-PyVenvConfig(
[String]
$ConfigDir
) {
Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg"
# Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue).
$pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue
# An empty map will be returned if no config file is found.
$pyvenvConfig = @{ }
if ($pyvenvConfigPath) {
Write-Verbose "File exists, parse `key = value` lines"
$pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath
$pyvenvConfigContent | ForEach-Object {
$keyval = $PSItem -split "\s*=\s*", 2
if ($keyval[0] -and $keyval[1]) {
$val = $keyval[1]
# Remove extraneous quotations around a string value.
if ("'""".Contains($val.Substring(0, 1))) {
$val = $val.Substring(1, $val.Length - 2)
}
$pyvenvConfig[$keyval[0]] = $val
Write-Verbose "Adding Key: '$($keyval[0])'='$val'"
}
}
}
return $pyvenvConfig
}
<# Begin Activate script --------------------------------------------------- #>
# Determine the containing directory of this script
$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition
$VenvExecDir = Get-Item -Path $VenvExecPath
Write-Verbose "Activation script is located in path: '$VenvExecPath'"
Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)"
Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)"
# Set values required in priority: CmdLine, ConfigFile, Default
# First, get the location of the virtual environment, it might not be
# VenvExecDir if specified on the command line.
if ($VenvDir) {
Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values"
}
else {
Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir."
$VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/")
Write-Verbose "VenvDir=$VenvDir"
}
# Next, read the `pyvenv.cfg` file to determine any required value such
# as `prompt`.
$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir
# Next, set the prompt from the command line, or the config file, or
# just use the name of the virtual environment folder.
if ($Prompt) {
Write-Verbose "Prompt specified as argument, using '$Prompt'"
}
else {
Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value"
if ($pyvenvCfg -and $pyvenvCfg['prompt']) {
Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'"
$Prompt = $pyvenvCfg['prompt'];
}
else {
Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virtual environment)"
Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'"
$Prompt = Split-Path -Path $venvDir -Leaf
}
}
Write-Verbose "Prompt = '$Prompt'"
Write-Verbose "VenvDir='$VenvDir'"
# Deactivate any currently active virtual environment, but leave the
# deactivate function in place.
deactivate -nondestructive
# Now set the environment variable VIRTUAL_ENV, used by many tools to determine
# that there is an activated venv.
$env:VIRTUAL_ENV = $VenvDir
if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) {
Write-Verbose "Setting prompt to '$Prompt'"
# Set the prompt to include the env name
# Make sure _OLD_VIRTUAL_PROMPT is global
function global:_OLD_VIRTUAL_PROMPT { "" }
Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT
New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt
function global:prompt {
Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) "
_OLD_VIRTUAL_PROMPT
}
$env:VIRTUAL_ENV_PROMPT = $Prompt
}
# Clear PYTHONHOME
if (Test-Path -Path Env:PYTHONHOME) {
Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME
Remove-Item -Path Env:PYTHONHOME
}
# Add the venv to the PATH
Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH
$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH"

69
venv/bin/activate Executable file
View File

@@ -0,0 +1,69 @@
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
# reset old environment variables
if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
PATH="${_OLD_VIRTUAL_PATH:-}"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi
if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
PS1="${_OLD_VIRTUAL_PS1:-}"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
unset VIRTUAL_ENV_PROMPT
if [ ! "${1:-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV=/home/kitraining/coco-tool/Abschluss-Projekt/venv
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/"bin":$PATH"
export PATH
# unset PYTHONHOME if set
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
# could use `if (set -u; : $PYTHONHOME) ;` in bash
if [ -n "${PYTHONHOME:-}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
_OLD_VIRTUAL_PS1="${PS1:-}"
PS1='(venv) '"${PS1:-}"
export PS1
VIRTUAL_ENV_PROMPT='(venv) '
export VIRTUAL_ENV_PROMPT
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
hash -r 2> /dev/null
fi

26
venv/bin/activate.csh Executable file
View File

@@ -0,0 +1,26 @@
# This file must be used with "source bin/activate.csh" *from csh*.
# You cannot run it directly.
# Created by Davide Di Blasi <davidedb@gmail.com>.
# Ported to Python 3.3 venv by Andrew Svetlov <andrew.svetlov@gmail.com>
alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; unsetenv VIRTUAL_ENV_PROMPT; test "\!:*" != "nondestructive" && unalias deactivate'
# Unset irrelevant variables.
deactivate nondestructive
setenv VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
set _OLD_VIRTUAL_PATH="$PATH"
setenv PATH "$VIRTUAL_ENV/"bin":$PATH"
set _OLD_VIRTUAL_PROMPT="$prompt"
if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then
set prompt = '(venv) '"$prompt"
setenv VIRTUAL_ENV_PROMPT '(venv) '
endif
alias pydoc python -m pydoc
rehash

69
venv/bin/activate.fish Executable file
View File

@@ -0,0 +1,69 @@
# This file must be used with "source <venv>/bin/activate.fish" *from fish*
# (https://fishshell.com/); you cannot run it directly.
function deactivate -d "Exit virtual environment and return to normal shell environment"
# reset old environment variables
if test -n "$_OLD_VIRTUAL_PATH"
set -gx PATH $_OLD_VIRTUAL_PATH
set -e _OLD_VIRTUAL_PATH
end
if test -n "$_OLD_VIRTUAL_PYTHONHOME"
set -gx PYTHONHOME $_OLD_VIRTUAL_PYTHONHOME
set -e _OLD_VIRTUAL_PYTHONHOME
end
if test -n "$_OLD_FISH_PROMPT_OVERRIDE"
set -e _OLD_FISH_PROMPT_OVERRIDE
# prevents error when using nested fish instances (Issue #93858)
if functions -q _old_fish_prompt
functions -e fish_prompt
functions -c _old_fish_prompt fish_prompt
functions -e _old_fish_prompt
end
end
set -e VIRTUAL_ENV
set -e VIRTUAL_ENV_PROMPT
if test "$argv[1]" != "nondestructive"
# Self-destruct!
functions -e deactivate
end
end
# Unset irrelevant variables.
deactivate nondestructive
set -gx VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
set -gx _OLD_VIRTUAL_PATH $PATH
set -gx PATH "$VIRTUAL_ENV/"bin $PATH
# Unset PYTHONHOME if set.
if set -q PYTHONHOME
set -gx _OLD_VIRTUAL_PYTHONHOME $PYTHONHOME
set -e PYTHONHOME
end
if test -z "$VIRTUAL_ENV_DISABLE_PROMPT"
# fish uses a function instead of an env var to generate the prompt.
# Save the current fish_prompt function as the function _old_fish_prompt.
functions -c fish_prompt _old_fish_prompt
# With the original prompt function renamed, we can override with our own.
function fish_prompt
# Save the return status of the last command.
set -l old_status $status
# Output the venv prompt; color taken from the blue of the Python logo.
printf "%s%s%s" (set_color 4B8BBE) '(venv) ' (set_color normal)
# Restore the return status of the previous command.
echo "exit $old_status" | .
# Output the original/"old" prompt.
_old_fish_prompt
end
set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV"
set -gx VIRTUAL_ENV_PROMPT '(venv) '
end

8
venv/bin/dotenv Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from dotenv.__main__ import cli
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(cli())

8
venv/bin/flask Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from flask.cli import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/normalizer Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from charset_normalizer.cli import cli_detect
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(cli_detect())

8
venv/bin/pip Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/pip3 Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

8
venv/bin/pip3.11 Executable file
View File

@@ -0,0 +1,8 @@
#!/home/kitraining/coco-tool/Abschluss-Projekt/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
sys.exit(main())

1
venv/bin/python Symbolic link
View File

@@ -0,0 +1 @@
python3

Some files were not shown because too many files have changed in this diff Show More