Compare commits
5 Commits
0e31237b79
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 14ad709b53 | |||
| ccfb40a2b3 | |||
|
|
036f3b178a | ||
|
|
5bfe82fc26 | ||
|
|
de5a8d2028 |
2
.gitignore
vendored
Normal file → Executable file
2
.gitignore
vendored
Normal file → Executable file
@@ -3,3 +3,5 @@ backend/uploads/*.pth
|
||||
*.pth
|
||||
backend/node_modules/
|
||||
backend/.venv/
|
||||
key
|
||||
key.pub
|
||||
@@ -1,217 +0,0 @@
|
||||
# Transfer Learning Base Configuration Feature
|
||||
|
||||
## Overview
|
||||
This feature implements automatic loading of base configurations when "Train on COCO" transfer learning is selected. Base parameters are loaded from `backend/data/` based on the selected YOLOX model, and these protected fields are displayed as greyed out and non-editable in the frontend.
|
||||
|
||||
## Components Modified/Created
|
||||
|
||||
### Backend
|
||||
|
||||
#### 1. Base Configuration Files (`backend/data/`)
|
||||
- **`yolox_s.py`** - Base config for YOLOX-Small (depth=0.33, width=0.50)
|
||||
- **`yolox_m.py`** - Base config for YOLOX-Medium (depth=0.67, width=0.75)
|
||||
- **`yolox_l.py`** - Base config for YOLOX-Large (depth=1.0, width=1.0)
|
||||
- **`yolox_x.py`** - Base config for YOLOX-XLarge (depth=1.33, width=1.25)
|
||||
|
||||
Each file contains a `BaseExp` class with protected parameters:
|
||||
- Model architecture (depth, width, activation)
|
||||
- Training hyperparameters (max_epoch, warmup_epochs, scheduler, etc.)
|
||||
- Optimizer settings (momentum, weight_decay)
|
||||
- Augmentation probabilities (mosaic_prob, mixup_prob, etc.)
|
||||
- Input/output sizes
|
||||
|
||||
#### 2. Services (`backend/services/generate_yolox_exp.py`)
|
||||
**New functions:**
|
||||
- `load_base_config(selected_model)` - Dynamically loads base config using importlib
|
||||
- Modified `generate_yolox_inference_exp()` to support `use_base_config` parameter
|
||||
- Base config merging logic: base → user overrides → defaults
|
||||
|
||||
**Behavior:**
|
||||
- `transfer_learning='coco'` → loads base config + applies user overrides
|
||||
- `transfer_learning='sketch'` → uses only user-defined values
|
||||
- Protected parameters from base config are preserved unless explicitly overridden
|
||||
|
||||
#### 3. API Routes (`backend/routes/api.py`)
|
||||
**New endpoint:**
|
||||
```python
|
||||
@api_bp.route('/base-config/<model_name>', methods=['GET'])
|
||||
def get_base_config(model_name):
|
||||
```
|
||||
Returns the base configuration JSON for a specific YOLOX model.
|
||||
|
||||
### Frontend
|
||||
|
||||
#### 1. HTML (`edit-training.html`)
|
||||
**Added:**
|
||||
- Info banner to indicate when base config is active
|
||||
- CSS styles for disabled input fields (grey background, not-allowed cursor)
|
||||
- Visual feedback showing which model's base config is loaded
|
||||
|
||||
**Banner HTML:**
|
||||
```html
|
||||
<div id="base-config-info" style="display:none; ...">
|
||||
🔒 Base Configuration Active
|
||||
Protected parameters are loaded from [model] base config
|
||||
</div>
|
||||
```
|
||||
|
||||
**CSS for disabled fields:**
|
||||
```css
|
||||
.setting-row input[type="number"]:disabled,
|
||||
.setting-row input[type="text"]:disabled,
|
||||
.setting-row input[type="checkbox"]:disabled {
|
||||
background: #d3d3d3 !important;
|
||||
color: #666 !important;
|
||||
cursor: not-allowed !important;
|
||||
border: 1px solid #999 !important;
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. JavaScript (`js/start-training.js`)
|
||||
**New functionality:**
|
||||
|
||||
1. **Base Config Loading:**
|
||||
```javascript
|
||||
function loadBaseConfig(modelName)
|
||||
```
|
||||
Fetches base config from `/api/base-config/<model>`
|
||||
|
||||
2. **Apply Base Config:**
|
||||
```javascript
|
||||
function applyBaseConfig(config, isCocoMode)
|
||||
```
|
||||
- Applies config values to form fields
|
||||
- Disables and greys out protected fields
|
||||
- Shows/hides info banner
|
||||
- Adds tooltips to disabled fields
|
||||
|
||||
3. **Update Transfer Learning Mode:**
|
||||
```javascript
|
||||
function updateTransferLearningMode()
|
||||
```
|
||||
- Monitors changes to "Transfer Learning" dropdown
|
||||
- Monitors changes to "Select Model" dropdown
|
||||
- Loads appropriate base config when COCO mode is selected
|
||||
- Clears base config when sketch mode is selected
|
||||
|
||||
4. **Form Submission Enhancement:**
|
||||
- Temporarily enables disabled fields before submission
|
||||
- Ensures protected parameters are included in form data
|
||||
- Re-disables fields after collection
|
||||
|
||||
**Protected Fields List:**
|
||||
```javascript
|
||||
const protectedFields = [
|
||||
'depth', 'width', 'act', 'max_epoch', 'warmup_epochs', 'warmup_lr',
|
||||
'scheduler', 'no_aug_epochs', 'min_lr_ratio', 'ema', 'weight_decay',
|
||||
'momentum', 'input_size', 'mosaic_scale', 'test_size', 'enable_mixup',
|
||||
'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob', 'degrees',
|
||||
'translate', 'shear', 'mixup_scale', 'print_interval', 'eval_interval'
|
||||
];
|
||||
```
|
||||
|
||||
## User Flow
|
||||
|
||||
### 1. Normal Custom Training (Train from sketch)
|
||||
- User selects model: e.g., "YOLOX-s"
|
||||
- User selects "Train from sketch"
|
||||
- All fields are editable (white background)
|
||||
- User can customize all parameters
|
||||
- Submission uses user-defined values only
|
||||
|
||||
### 2. COCO Transfer Learning (Train on COCO)
|
||||
- User selects model: e.g., "YOLOX-s"
|
||||
- User selects "Train on coco"
|
||||
- **Automatic actions:**
|
||||
1. Frontend calls `/api/base-config/YOLOX-s`
|
||||
2. Base config is loaded and applied
|
||||
3. Protected fields become greyed out and disabled
|
||||
4. Green info banner appears: "🔒 Base Configuration Active"
|
||||
5. Tooltip on hover: "Protected by base config for YOLOX-s. Switch to 'Train from sketch' to customize."
|
||||
- User can still edit non-protected fields
|
||||
- On submit: both base config values AND user overrides are sent to backend
|
||||
- Backend generates exp.py with merged settings
|
||||
|
||||
### 3. Switching Models
|
||||
- User changes from "YOLOX-s" to "YOLOX-l" (while in COCO mode)
|
||||
- Frontend automatically:
|
||||
1. Fetches new base config for YOLOX-l
|
||||
2. Updates field values (depth=1.0, width=1.0, etc.)
|
||||
3. Updates banner text to show "YOLOX-l"
|
||||
- Protected parameters update to match new model's architecture
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Test Steps:
|
||||
|
||||
1. **Test Base Config Loading:**
|
||||
```bash
|
||||
cd backend/data
|
||||
python test_base_configs.py
|
||||
```
|
||||
Should display all parameters for yolox-s, yolox-m, yolox-l, yolox-x
|
||||
|
||||
2. **Test API Endpoint:**
|
||||
```bash
|
||||
# Start Flask server
|
||||
cd backend
|
||||
python app.py
|
||||
|
||||
# In another terminal:
|
||||
curl http://localhost:3000/api/base-config/YOLOX-s
|
||||
```
|
||||
Should return JSON with depth, width, activation, etc.
|
||||
|
||||
3. **Test Frontend:**
|
||||
- Open `edit-training.html?id=<project_id>` in browser
|
||||
- Select "YOLOX-s" model
|
||||
- Select "Train on coco" → fields should grey out
|
||||
- Select "Train from sketch" → fields should become editable
|
||||
- Switch to "YOLOX-l" (in COCO mode) → values should update
|
||||
- Open browser console and check for: `Applied base config. Protected fields: depth, width, ...`
|
||||
|
||||
4. **Test Form Submission:**
|
||||
- With COCO mode active (fields greyed out)
|
||||
- Click "Save Parameters"
|
||||
- Check browser Network tab → POST to `/api/yolox-settings`
|
||||
- Verify payload includes protected parameters (depth, width, etc.)
|
||||
- Check Flask logs for successful save
|
||||
|
||||
### Expected Behaviors:
|
||||
|
||||
✅ **COCO mode + YOLOX-s:**
|
||||
- depth: 0.33 (greyed out)
|
||||
- width: 0.50 (greyed out)
|
||||
- activation: silu (greyed out)
|
||||
- Info banner visible
|
||||
|
||||
✅ **COCO mode + YOLOX-l:**
|
||||
- depth: 1.0 (greyed out)
|
||||
- width: 1.0 (greyed out)
|
||||
- activation: silu (greyed out)
|
||||
|
||||
✅ **Sketch mode:**
|
||||
- All fields white/editable
|
||||
- No info banner
|
||||
- User can set any values
|
||||
|
||||
## Documentation
|
||||
|
||||
- **`backend/data/README.md`** - Complete guide on base config system
|
||||
- **`backend/data/test_base_configs.py`** - Test script for base configs
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Proven defaults:** Users start with battle-tested COCO pretraining settings
|
||||
2. **Prevents mistakes:** Can't accidentally break model architecture by changing depth/width
|
||||
3. **Easy customization:** Can still override specific parameters if needed
|
||||
4. **Visual feedback:** Clear indication of which fields are protected
|
||||
5. **Model-specific:** Each model (s/m/l/x) has appropriate architecture defaults
|
||||
6. **Flexible:** Can easily add new models by creating new base config files
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Add "Override" button next to protected fields to unlock individual parameters
|
||||
- Show diff comparison between base config and user overrides
|
||||
- Add validation warnings if user tries values far from base config ranges
|
||||
- Export final merged config as preview before training
|
||||
124
add-project.html
Normal file → Executable file
124
add-project.html
Normal file → Executable file
@@ -29,6 +29,14 @@
|
||||
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover=""
|
||||
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" class="button-red">Add Training Project</button>
|
||||
<button id="Add Dataset" class="button">Add Dataset</button>
|
||||
<button id="Import Dataset" class="button">Refresh Label-Studio</button>
|
||||
@@ -234,7 +242,123 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus();
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
|
||||
</body>
|
||||
|
||||
|
||||
0
backend/.gitignore
vendored
Normal file → Executable file
0
backend/.gitignore
vendored
Normal file → Executable file
@@ -1,28 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate/"
|
||||
self.train_ann = "coco_project_27_train.json"
|
||||
self.val_ann = "coco_project_27_valid.json"
|
||||
self.test_ann = "coco_project_27_test.json"
|
||||
self.num_classes = 80
|
||||
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
|
||||
|
||||
|
||||
self.depth = 1.0
|
||||
self.width = 1.0
|
||||
self.input_size = (640.0, 640.0)
|
||||
self.mosaic_scale = (0.1, 2.0)
|
||||
self.random_size = (10, 20)
|
||||
self.test_size = (640.0, 640.0)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
self.enable_mixup = False
|
||||
@@ -1,48 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate/"
|
||||
self.train_ann = "coco_project_2_train.json"
|
||||
self.val_ann = "coco_project_2_valid.json"
|
||||
self.test_ann = "coco_project_2_test.json"
|
||||
self.num_classes = 2
|
||||
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
|
||||
self.activation = "silu"
|
||||
self.depth = 0.33
|
||||
self.scheduler = "yoloxwarmcos"
|
||||
self.width = 0.5
|
||||
self.input_size = (640.0, 640.0)
|
||||
self.mosaic_scale = (0.1, 2.0)
|
||||
self.test_size = (640.0, 640.0)
|
||||
self.enable_mixup = True
|
||||
self.max_epoch = 300
|
||||
self.warmup_epochs = 5
|
||||
self.warmup_lr = 0.0
|
||||
self.no_aug_epochs = 15
|
||||
self.min_lr_ratio = 0.05
|
||||
self.ema = True
|
||||
self.weight_decay = 0.0005
|
||||
self.momentum = 0.9
|
||||
self.print_interval = 10
|
||||
self.eval_interval = 10
|
||||
self.test_conf = 0.01
|
||||
self.nms_thre = 0.65
|
||||
self.mosaic_prob = 1.0
|
||||
self.mixup_prob = 1.0
|
||||
self.hsv_prob = 1.0
|
||||
self.flip_prob = 0.5
|
||||
self.degrees = 10.0
|
||||
self.translate = 0.1
|
||||
self.shear = 2.0
|
||||
self.mixup_scale = (0.5, 1.5)
|
||||
self.random_size = (10, 20)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
@@ -1,48 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate/"
|
||||
self.train_ann = "coco_project_6_train.json"
|
||||
self.val_ann = "coco_project_6_valid.json"
|
||||
self.test_ann = "coco_project_6_test.json"
|
||||
self.num_classes = 2
|
||||
self.pretrained_ckpt = r'/home/kitraining/Yolox/YOLOX-main/pretrained/YOLOX-s.pth'
|
||||
self.depth = 0.33
|
||||
self.width = 0.5
|
||||
self.input_size = (640.0, 640.0)
|
||||
self.mosaic_scale = (0.1, 2.0)
|
||||
self.test_size = (640.0, 640.0)
|
||||
self.enable_mixup = True
|
||||
self.max_epoch = 300
|
||||
self.warmup_epochs = 5
|
||||
self.warmup_lr = 0.0
|
||||
self.scheduler = "yoloxwarmcos"
|
||||
self.no_aug_epochs = 15
|
||||
self.min_lr_ratio = 0.05
|
||||
self.ema = True
|
||||
self.weight_decay = 0.0005
|
||||
self.momentum = 0.9
|
||||
self.print_interval = 10
|
||||
self.eval_interval = 10
|
||||
self.test_conf = 0.01
|
||||
self.nms_thre = 0.65
|
||||
self.mosaic_prob = 1.0
|
||||
self.mixup_prob = 1.0
|
||||
self.hsv_prob = 1.0
|
||||
self.flip_prob = 0.5
|
||||
self.degrees = 10.0
|
||||
self.translate = 0.1
|
||||
self.shear = 2.0
|
||||
self.mixup_scale = (0.5, 1.5)
|
||||
self.activation = "silu"
|
||||
self.random_size = (10, 20)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
@@ -1,113 +0,0 @@
|
||||
# Quick Start Guide - Python Backend
|
||||
|
||||
## Step-by-Step Setup
|
||||
|
||||
### 1. Install Python
|
||||
Make sure you have Python 3.8 or higher installed:
|
||||
```bash
|
||||
python --version
|
||||
```
|
||||
|
||||
### 2. Create Virtual Environment
|
||||
```bash
|
||||
cd backend
|
||||
python -m venv venv
|
||||
```
|
||||
|
||||
### 3. Activate Virtual Environment
|
||||
|
||||
**Windows:**
|
||||
```powershell
|
||||
.\venv\Scripts\Activate.ps1
|
||||
```
|
||||
|
||||
**Linux/Mac:**
|
||||
```bash
|
||||
source venv/bin/activate
|
||||
```
|
||||
|
||||
### 4. Install Dependencies
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 5. Verify Database Connection
|
||||
Make sure MySQL is running and the database `myapp` exists:
|
||||
```sql
|
||||
CREATE DATABASE IF NOT EXISTS myapp;
|
||||
```
|
||||
|
||||
### 6. Run the Server
|
||||
```bash
|
||||
python start.py
|
||||
```
|
||||
|
||||
Or:
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
The server should now be running at `http://0.0.0.0:3000`
|
||||
|
||||
## Testing the API
|
||||
|
||||
Test if the server is working:
|
||||
```bash
|
||||
curl http://localhost:3000/api/training-projects
|
||||
```
|
||||
|
||||
## Common Issues
|
||||
|
||||
### ModuleNotFoundError
|
||||
If you get import errors, make sure you've activated the virtual environment and installed all dependencies.
|
||||
|
||||
### Database Connection Error
|
||||
Check that:
|
||||
- MySQL is running
|
||||
- Database credentials in `app.py` are correct
|
||||
- Database `myapp` exists
|
||||
|
||||
### Port Already in Use
|
||||
If port 3000 is already in use, modify the port in `app.py`:
|
||||
```python
|
||||
app.run(host='0.0.0.0', port=3001, debug=True)
|
||||
```
|
||||
|
||||
## What Changed from Node.js
|
||||
|
||||
1. **Server Framework**: Express.js → Flask
|
||||
2. **ORM**: Sequelize → SQLAlchemy
|
||||
3. **HTTP Client**: node-fetch → requests
|
||||
4. **Package Manager**: npm → pip
|
||||
5. **Dependencies**: package.json → requirements.txt
|
||||
6. **Startup**: `node server.js` → `python app.py`
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Test all API endpoints
|
||||
2. Update frontend to point to the new Python backend (if needed)
|
||||
3. Migrate any remaining Node.js-specific logic
|
||||
4. Test file uploads and downloads
|
||||
5. Test YOLOX training functionality
|
||||
|
||||
## File Structure Comparison
|
||||
|
||||
**Before (Node.js):**
|
||||
```
|
||||
backend/
|
||||
├── server.js
|
||||
├── package.json
|
||||
├── routes/api.js
|
||||
├── models/*.js
|
||||
└── services/*.js
|
||||
```
|
||||
|
||||
**After (Python):**
|
||||
```
|
||||
backend/
|
||||
├── app.py
|
||||
├── requirements.txt
|
||||
├── routes/api.py
|
||||
├── models/*.py
|
||||
└── services/*.py
|
||||
```
|
||||
0
backend/README.md
Normal file → Executable file
0
backend/README.md
Normal file → Executable file
0
backend/app.py
Normal file → Executable file
0
backend/app.py
Normal file → Executable file
@@ -1,25 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate/"
|
||||
self.train_ann = "coco_project_5_train.json"
|
||||
self.val_ann = "coco_project_5_valid.json"
|
||||
self.test_ann = "coco_project_5_test.json"
|
||||
self.num_classes = 4
|
||||
self.depth = 1.0
|
||||
self.width = 1.0
|
||||
self.input_size = (640, 640)
|
||||
self.mosaic_scale = (0.1, 2)
|
||||
self.random_size = (10, 20)
|
||||
self.test_size = (640, 640)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
self.enable_mixup = False
|
||||
@@ -1,84 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate" # Where images are located
|
||||
self.annotations_dir = "./backend/1/custom_exp_1" # Where annotation JSONs are located
|
||||
self.train_ann = "coco_project_1_train.json"
|
||||
self.val_ann = "coco_project_1_valid.json"
|
||||
self.test_ann = "coco_project_1_test.json"
|
||||
self.num_classes = 2
|
||||
# Disable train2017 subdirectory - our images are directly in data_dir
|
||||
self.name = ""
|
||||
# Set data workers for training
|
||||
self.data_num_workers = 8
|
||||
self.depth = 1.0
|
||||
self.width = 1.0
|
||||
self.input_size = (640.0, 640.0)
|
||||
self.mosaic_scale = (0.1, 2.0)
|
||||
self.test_size = (640.0, 640.0)
|
||||
self.enable_mixup = True
|
||||
self.max_epoch = 300
|
||||
self.warmup_epochs = 5
|
||||
self.warmup_lr = 0.0
|
||||
self.scheduler = "yoloxwarmcos"
|
||||
self.no_aug_epochs = 15
|
||||
self.min_lr_ratio = 0.05
|
||||
self.ema = True
|
||||
self.weight_decay = 0.0005
|
||||
self.momentum = 0.9
|
||||
self.print_interval = 10
|
||||
self.eval_interval = 10
|
||||
self.test_conf = 0.01
|
||||
self.nms_thre = 0.65
|
||||
self.mosaic_prob = 1.0
|
||||
self.mixup_prob = 1.0
|
||||
self.hsv_prob = 1.0
|
||||
self.flip_prob = 0.5
|
||||
self.degrees = 10.0
|
||||
self.translate = 0.1
|
||||
self.shear = 2.0
|
||||
self.mixup_scale = (0.5, 1.5)
|
||||
self.activation = "silu"
|
||||
self.random_size = (10, 20)
|
||||
|
||||
def get_dataset(self, cache=False, cache_type="ram"):
|
||||
"""Override to use name parameter for images directory"""
|
||||
from yolox.data import COCODataset
|
||||
|
||||
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
|
||||
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
|
||||
# Use empty string for name since we have absolute paths in JSON
|
||||
return COCODataset(
|
||||
data_dir=self.annotations_dir,
|
||||
json_file=self.train_ann,
|
||||
name="",
|
||||
img_size=self.input_size,
|
||||
preproc=self.preproc if hasattr(self, 'preproc') else None,
|
||||
cache=cache,
|
||||
cache_type=cache_type,
|
||||
)
|
||||
|
||||
def get_eval_dataset(self, **kwargs):
|
||||
"""Override eval dataset using name parameter"""
|
||||
from yolox.data import COCODataset
|
||||
|
||||
testdev = kwargs.get("testdev", False)
|
||||
legacy = kwargs.get("legacy", False)
|
||||
|
||||
return COCODataset(
|
||||
data_dir=self.annotations_dir,
|
||||
json_file=self.val_ann if not testdev else self.test_ann,
|
||||
name="",
|
||||
img_size=self.test_size,
|
||||
preproc=None, # No preprocessing for evaluation
|
||||
)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,84 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
# Copyright (c) Megvii, Inc. and its affiliates.
|
||||
|
||||
import os
|
||||
|
||||
from yolox.exp import Exp as MyExp
|
||||
|
||||
|
||||
class Exp(MyExp):
|
||||
def __init__(self):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "/home/kitraining/To_Annotate" # Where images are located
|
||||
self.annotations_dir = "./backend/1/custom_exp_1" # Where annotation JSONs are located
|
||||
self.train_ann = "coco_project_1_train.json"
|
||||
self.val_ann = "coco_project_1_valid.json"
|
||||
self.test_ann = "coco_project_1_test.json"
|
||||
self.num_classes = 2
|
||||
# Disable train2017 subdirectory - our images are directly in data_dir
|
||||
self.name = ""
|
||||
# Set data workers for training
|
||||
self.data_num_workers = 8
|
||||
self.depth = 1.0
|
||||
self.width = 1.0
|
||||
self.input_size = (640.0, 640.0)
|
||||
self.mosaic_scale = (0.1, 2.0)
|
||||
self.test_size = (640.0, 640.0)
|
||||
self.enable_mixup = True
|
||||
self.max_epoch = 300
|
||||
self.warmup_epochs = 5
|
||||
self.warmup_lr = 0.0
|
||||
self.scheduler = "yoloxwarmcos"
|
||||
self.no_aug_epochs = 15
|
||||
self.min_lr_ratio = 0.05
|
||||
self.ema = True
|
||||
self.weight_decay = 0.0005
|
||||
self.momentum = 0.9
|
||||
self.print_interval = 10
|
||||
self.eval_interval = 10
|
||||
self.test_conf = 0.01
|
||||
self.nms_thre = 0.65
|
||||
self.mosaic_prob = 1.0
|
||||
self.mixup_prob = 1.0
|
||||
self.hsv_prob = 1.0
|
||||
self.flip_prob = 0.5
|
||||
self.degrees = 10.0
|
||||
self.translate = 0.1
|
||||
self.shear = 2.0
|
||||
self.mixup_scale = (0.5, 1.5)
|
||||
self.activation = "silu"
|
||||
self.random_size = (10, 20)
|
||||
|
||||
def get_dataset(self, cache=False, cache_type="ram"):
|
||||
"""Override to use name parameter for images directory"""
|
||||
from yolox.data import COCODataset
|
||||
|
||||
# COCODataset constructs image paths as: os.path.join(data_dir, name, file_name)
|
||||
# YOLOX adds "annotations/" to data_dir automatically, so we pass annotations_dir directly
|
||||
# Use empty string for name since we have absolute paths in JSON
|
||||
return COCODataset(
|
||||
data_dir=self.annotations_dir,
|
||||
json_file=self.train_ann,
|
||||
name="",
|
||||
img_size=self.input_size,
|
||||
preproc=self.preproc if hasattr(self, 'preproc') else None,
|
||||
cache=cache,
|
||||
cache_type=cache_type,
|
||||
)
|
||||
|
||||
def get_eval_dataset(self, **kwargs):
|
||||
"""Override eval dataset using name parameter"""
|
||||
from yolox.data import COCODataset
|
||||
|
||||
testdev = kwargs.get("testdev", False)
|
||||
legacy = kwargs.get("legacy", False)
|
||||
|
||||
return COCODataset(
|
||||
data_dir=self.annotations_dir,
|
||||
json_file=self.val_ann if not testdev else self.test_ann,
|
||||
name="",
|
||||
img_size=self.test_size,
|
||||
preproc=None, # No preprocessing for evaluation
|
||||
)
|
||||
self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]
|
||||
0
backend/check_db.py
Normal file → Executable file
0
backend/check_db.py
Normal file → Executable file
0
backend/data/README.md
Normal file → Executable file
0
backend/data/README.md
Normal file → Executable file
0
backend/data/__init__.py
Normal file → Executable file
0
backend/data/__init__.py
Normal file → Executable file
0
backend/data/test_base_configs.py
Normal file → Executable file
0
backend/data/test_base_configs.py
Normal file → Executable file
0
backend/data/yolox_l.py
Normal file → Executable file
0
backend/data/yolox_l.py
Normal file → Executable file
0
backend/data/yolox_m.py
Normal file → Executable file
0
backend/data/yolox_m.py
Normal file → Executable file
0
backend/data/yolox_s.py
Normal file → Executable file
0
backend/data/yolox_s.py
Normal file → Executable file
0
backend/data/yolox_x.py
Normal file → Executable file
0
backend/data/yolox_x.py
Normal file → Executable file
0
backend/database/__init__.py
Normal file → Executable file
0
backend/database/__init__.py
Normal file → Executable file
@@ -1,10 +0,0 @@
|
||||
// database.js
|
||||
const { Sequelize } = require('sequelize');
|
||||
|
||||
const sequelize = new Sequelize('myapp', 'root', 'root', {
|
||||
host: 'localhost',
|
||||
dialect: 'mysql',
|
||||
logging: false,
|
||||
});
|
||||
|
||||
module.exports = sequelize;
|
||||
0
backend/database/database.py
Normal file → Executable file
0
backend/database/database.py
Normal file → Executable file
@@ -1,12 +0,0 @@
|
||||
-- Migration: Add width and height columns to image table
|
||||
-- Date: 2025-11-27
|
||||
|
||||
USE myapp;
|
||||
|
||||
-- Add width and height columns to image table
|
||||
ALTER TABLE `image`
|
||||
ADD COLUMN `width` FLOAT NULL AFTER `image_path`,
|
||||
ADD COLUMN `height` FLOAT NULL AFTER `width`;
|
||||
|
||||
-- Verify the changes
|
||||
DESCRIBE `image`;
|
||||
@@ -1,12 +0,0 @@
|
||||
-- Migration to change width and height from FLOAT to INT in image table
|
||||
-- Run this after updating the Images model
|
||||
|
||||
-- First, backup the table (optional but recommended)
|
||||
-- CREATE TABLE image_backup AS SELECT * FROM image;
|
||||
|
||||
-- Alter the columns to INT type
|
||||
ALTER TABLE image MODIFY COLUMN width INT;
|
||||
ALTER TABLE image MODIFY COLUMN height INT;
|
||||
|
||||
-- Verify the changes
|
||||
DESCRIBE image;
|
||||
File diff suppressed because one or more lines are too long
@@ -1,40 +0,0 @@
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
|
||||
const Annotation = sequelize.define('Annotation', {
|
||||
annotation_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
primaryKey: true,
|
||||
autoIncrement: true,
|
||||
},
|
||||
image_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
allowNull: false,
|
||||
},
|
||||
x: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: false,
|
||||
},
|
||||
y: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: false,
|
||||
},
|
||||
height: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: false,
|
||||
},
|
||||
width: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: false,
|
||||
},
|
||||
Label: {
|
||||
type: DataTypes.STRING,
|
||||
allowNull: false,
|
||||
},
|
||||
}, {
|
||||
tableName: 'annotation',
|
||||
timestamps: false,
|
||||
});
|
||||
|
||||
module.exports = Annotation;
|
||||
0
backend/models/Annotation.py
Normal file → Executable file
0
backend/models/Annotation.py
Normal file → Executable file
0
backend/models/AnnotationProjectMapping.py
Normal file → Executable file
0
backend/models/AnnotationProjectMapping.py
Normal file → Executable file
0
backend/models/ClassMapping.py
Normal file → Executable file
0
backend/models/ClassMapping.py
Normal file → Executable file
@@ -1,35 +0,0 @@
|
||||
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
const Image = sequelize.define('Image', {
|
||||
image_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
primaryKey: true,
|
||||
autoIncrement: true,
|
||||
},
|
||||
image_path: {
|
||||
type: DataTypes.STRING,
|
||||
allowNull: false,
|
||||
},
|
||||
project_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
allowNull: false,
|
||||
},
|
||||
width: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: true,
|
||||
},
|
||||
height: {
|
||||
type: DataTypes.FLOAT,
|
||||
allowNull: true,
|
||||
},
|
||||
|
||||
}, {
|
||||
tableName: 'image',
|
||||
timestamps: false,
|
||||
});
|
||||
|
||||
module.exports = Image;
|
||||
|
||||
|
||||
0
backend/models/Images.py
Normal file → Executable file
0
backend/models/Images.py
Normal file → Executable file
@@ -1,24 +0,0 @@
|
||||
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
const Label_studio_project = sequelize.define('LabelStudioProject', {
|
||||
project_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
primaryKey: true,
|
||||
unique: true,
|
||||
allowNull: false,
|
||||
},
|
||||
title:{
|
||||
type: DataTypes.STRING,
|
||||
allowNull: false,
|
||||
}
|
||||
|
||||
}, {
|
||||
tableName: 'label_studio_project',
|
||||
timestamps: false,
|
||||
});
|
||||
|
||||
module.exports = Label_studio_project;
|
||||
|
||||
|
||||
0
backend/models/LabelStudioProject.py
Normal file → Executable file
0
backend/models/LabelStudioProject.py
Normal file → Executable file
0
backend/models/ProjectClass.py
Normal file → Executable file
0
backend/models/ProjectClass.py
Normal file → Executable file
0
backend/models/Settings.py
Normal file → Executable file
0
backend/models/Settings.py
Normal file → Executable file
@@ -1,38 +0,0 @@
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
const Training_Project = sequelize.define('LabelStudioProject', {
|
||||
project_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
primaryKey: true,
|
||||
unique: true,
|
||||
allowNull: false,
|
||||
autoIncrement: true,
|
||||
},
|
||||
title:{
|
||||
type: DataTypes.STRING,
|
||||
allowNull: false,
|
||||
},
|
||||
description: {
|
||||
type: DataTypes.STRING,
|
||||
},
|
||||
classes: {
|
||||
type: DataTypes.JSON,
|
||||
allowNull: false,
|
||||
},
|
||||
project_image: {
|
||||
type: DataTypes.BLOB,
|
||||
},
|
||||
project_image_type: {
|
||||
type: DataTypes.STRING,
|
||||
allowNull: true,
|
||||
}
|
||||
|
||||
}, {
|
||||
tableName: 'training_project',
|
||||
timestamps: false,
|
||||
});
|
||||
|
||||
module.exports = Training_Project;
|
||||
|
||||
|
||||
0
backend/models/TrainingProject.py
Normal file → Executable file
0
backend/models/TrainingProject.py
Normal file → Executable file
@@ -1,33 +0,0 @@
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
const TrainingProjectDetails = sequelize.define('TrainingProjectDetails', {
|
||||
id: {
|
||||
type: DataTypes.INTEGER,
|
||||
primaryKey: true,
|
||||
autoIncrement: true,
|
||||
unique: true,
|
||||
},
|
||||
project_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
allowNull: false,
|
||||
unique: true,
|
||||
},
|
||||
annotation_projects: {
|
||||
type: DataTypes.JSON,
|
||||
allowNull: false,
|
||||
},
|
||||
class_map: {
|
||||
type: DataTypes.JSON,
|
||||
allowNull: true,
|
||||
},
|
||||
description: {
|
||||
type: DataTypes.JSON,
|
||||
allowNull: true,
|
||||
}
|
||||
}, {
|
||||
tableName: 'training_project_details',
|
||||
timestamps: false,
|
||||
});
|
||||
|
||||
module.exports = TrainingProjectDetails;
|
||||
0
backend/models/TrainingProjectDetails.py
Normal file → Executable file
0
backend/models/TrainingProjectDetails.py
Normal file → Executable file
0
backend/models/TrainingSize.py
Normal file → Executable file
0
backend/models/TrainingSize.py
Normal file → Executable file
0
backend/models/__init__.py
Normal file → Executable file
0
backend/models/__init__.py
Normal file → Executable file
@@ -1,30 +0,0 @@
|
||||
const LabelStudioProject = require('./LabelStudioProject.js');
|
||||
const Annotation = require('./Annotation.js');
|
||||
const Image = require('./Images.js');
|
||||
const sequelize = require('../database/database.js');
|
||||
const TrainingProjectDetails = require('./TrainingProjectDetails.js');
|
||||
const TrainingProject = require('./TrainingProject.js');
|
||||
const Training = require('./training.js');
|
||||
|
||||
|
||||
|
||||
const Project = LabelStudioProject;
|
||||
const Img = Image;
|
||||
const Ann = Annotation;
|
||||
|
||||
// Associations
|
||||
Project.hasMany(Img, { foreignKey: 'project_id' });
|
||||
Img.belongsTo(Project, { foreignKey: 'project_id' });
|
||||
|
||||
Img.hasMany(Ann, { foreignKey: 'image_id' });
|
||||
Ann.belongsTo(Img, { foreignKey: 'image_id' });
|
||||
|
||||
// TrainingProjectDetails <-> TrainingProject
|
||||
TrainingProjectDetails.belongsTo(TrainingProject, { foreignKey: 'project_id' });
|
||||
TrainingProject.hasOne(TrainingProjectDetails, { foreignKey: 'project_id' });
|
||||
|
||||
// Training <-> TrainingProjectDetails
|
||||
Training.belongsTo(TrainingProjectDetails, { foreignKey: 'project_details_id' });
|
||||
TrainingProjectDetails.hasMany(Training, { foreignKey: 'project_details_id' });
|
||||
|
||||
module.exports = { Project, Img, Ann, TrainingProjectDetails, TrainingProject, Training };
|
||||
@@ -1,140 +0,0 @@
|
||||
const { DataTypes } = require('sequelize');
|
||||
const sequelize = require('../database/database.js');
|
||||
|
||||
const Training = sequelize.define('training', {
|
||||
id: {
|
||||
type: DataTypes.INTEGER,
|
||||
autoIncrement: true,
|
||||
unique: true,
|
||||
primaryKey: true
|
||||
},
|
||||
exp_name: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
max_epoch: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
depth: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
width: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
activation: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
warmup_epochs: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
warmup_lr: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
basic_lr_per_img: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
scheduler: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
no_aug_epochs: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
min_lr_ratio: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
ema: {
|
||||
type: DataTypes.BOOLEAN
|
||||
},
|
||||
weight_decay: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
momentum: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
input_size: {
|
||||
type: DataTypes.JSON
|
||||
},
|
||||
print_interval: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
eval_interval: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
save_history_ckpt: {
|
||||
type: DataTypes.BOOLEAN
|
||||
},
|
||||
test_size: {
|
||||
type: DataTypes.JSON
|
||||
},
|
||||
test_conf: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
nms_thre: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
multiscale_range: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
enable_mixup: {
|
||||
type: DataTypes.BOOLEAN
|
||||
},
|
||||
mosaic_prob: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
mixup_prob: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
hsv_prob: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
flip_prob: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
degrees: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
mosaic_scale: {
|
||||
type: DataTypes.JSON
|
||||
},
|
||||
mixup_scale: {
|
||||
type: DataTypes.JSON
|
||||
},
|
||||
translate: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
shear: {
|
||||
type: DataTypes.FLOAT
|
||||
},
|
||||
training_name: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
project_details_id: {
|
||||
type: DataTypes.INTEGER,
|
||||
allowNull: false
|
||||
},
|
||||
seed: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
train: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
valid: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
test: {
|
||||
type: DataTypes.INTEGER
|
||||
},
|
||||
selected_model: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
transfer_learning: {
|
||||
type: DataTypes.STRING(255)
|
||||
},
|
||||
model_upload: {
|
||||
type: DataTypes.BLOB
|
||||
}
|
||||
}, {
|
||||
tableName: 'training',
|
||||
timestamps: false
|
||||
});
|
||||
|
||||
module.exports = Training;
|
||||
0
backend/models/training.py
Normal file → Executable file
0
backend/models/training.py
Normal file → Executable file
0
backend/node
Normal file → Executable file
0
backend/node
Normal file → Executable file
0
backend/package-lock.json
generated
Normal file → Executable file
0
backend/package-lock.json
generated
Normal file → Executable file
0
backend/package.json
Normal file → Executable file
0
backend/package.json
Normal file → Executable file
0
backend/requirements.txt
Normal file → Executable file
0
backend/requirements.txt
Normal file → Executable file
0
backend/routes/__init__.py
Normal file → Executable file
0
backend/routes/__init__.py
Normal file → Executable file
@@ -1,496 +0,0 @@
|
||||
const express = require('express');
|
||||
const multer = require('multer');
|
||||
const upload = multer();
|
||||
const TrainingProject = require('../models/TrainingProject.js');
|
||||
const LabelStudioProject = require('../models/LabelStudioProject.js')
|
||||
const { seedLabelStudio, updateStatus } = require('../services/seed-label-studio.js');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const {generateTrainingJson} = require('../services/generate-json-yolox.js')
|
||||
|
||||
|
||||
const router = express.Router();
|
||||
|
||||
// Ensure JSON bodies are parsed for all routes
|
||||
router.use(express.json());
|
||||
|
||||
router.get('/seed', async (req, res) => {
|
||||
const result = await seedLabelStudio();
|
||||
res.json(result);
|
||||
});
|
||||
|
||||
|
||||
|
||||
// Trigger generate-json-yolox.js
|
||||
|
||||
router.post('/generate-yolox-json', async (req, res) => {
|
||||
const { project_id } = req.body;
|
||||
if (!project_id) {
|
||||
return res.status(400).json({ message: 'Missing project_id in request body' });
|
||||
}
|
||||
try {
|
||||
// Generate COCO JSONs
|
||||
// Find all TrainingProjectDetails for this project
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
|
||||
if (!detailsRows || detailsRows.length === 0) {
|
||||
return res.status(404).json({ message: 'No TrainingProjectDetails found for project ' + project_id });
|
||||
}
|
||||
// For each details row, generate coco.jsons and exp.py in projectfolder/project_details_id
|
||||
const Training = require('../models/training.js');
|
||||
const { saveYoloxExp } = require('../services/generate-yolox-exp.js');
|
||||
const TrainingProject = require('../models/TrainingProject.js');
|
||||
const trainingProject = await TrainingProject.findByPk(project_id);
|
||||
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
|
||||
for (const details of detailsRows) {
|
||||
const detailsId = details.id;
|
||||
await generateTrainingJson(detailsId);
|
||||
const trainings = await Training.findAll({ where: { project_details_id: detailsId } });
|
||||
if (trainings.length === 0) continue;
|
||||
// For each training, save exp.py in projectfolder/project_details_id
|
||||
const outDir = path.join(__dirname, '..', projectName, String(detailsId));
|
||||
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
|
||||
for (const training of trainings) {
|
||||
const expFilePath = path.join(outDir, 'exp.py');
|
||||
await saveYoloxExp(training.id, expFilePath);
|
||||
}
|
||||
}
|
||||
|
||||
// Find all trainings for this project
|
||||
// ...existing code...
|
||||
res.json({ message: 'YOLOX JSON and exp.py generated for project ' + project_id });
|
||||
} catch (err) {
|
||||
console.error('Error generating YOLOX JSON:', err);
|
||||
res.status(500).json({ message: 'Failed to generate YOLOX JSON', error: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Start YOLOX training
|
||||
const { spawn } = require('child_process');
|
||||
router.post('/start-yolox-training', async (req, res) => {
|
||||
try {
|
||||
const { project_id, training_id } = req.body;
|
||||
// Get project name
|
||||
const trainingProject = await TrainingProject.findByPk(project_id);
|
||||
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
|
||||
// Look up training row by id or project_details_id
|
||||
const Training = require('../models/training.js');
|
||||
let trainingRow = await Training.findByPk(training_id);
|
||||
if (!trainingRow) {
|
||||
trainingRow = await Training.findOne({ where: { project_details_id: training_id } });
|
||||
}
|
||||
if (!trainingRow) {
|
||||
return res.status(404).json({ error: `Training row not found for id or project_details_id ${training_id}` });
|
||||
}
|
||||
const project_details_id = trainingRow.project_details_id;
|
||||
// Use the generated exp.py from the correct project folder
|
||||
const outDir = path.join(__dirname, '..', projectName, String(project_details_id));
|
||||
const yoloxMainDir = '/home/kitraining/Yolox/YOLOX-main';
|
||||
const expSrc = path.join(outDir, 'exp.py');
|
||||
if (!fs.existsSync(expSrc)) {
|
||||
return res.status(500).json({ error: `exp.py not found at ${expSrc}` });
|
||||
}
|
||||
// Source venv and run YOLOX training in YOLOX-main folder
|
||||
const yoloxVenv = '/home/kitraining/Yolox/yolox_venv/bin/activate';
|
||||
// Determine model argument based on selected_model and transfer_learning
|
||||
let modelArg = '';
|
||||
let cmd = '';
|
||||
if (
|
||||
trainingRow.transfer_learning &&
|
||||
typeof trainingRow.transfer_learning === 'string' &&
|
||||
trainingRow.transfer_learning.toLowerCase() === 'coco'
|
||||
) {
|
||||
// If transfer_learning is 'coco', add -o and modelArg
|
||||
modelArg = ` -c /home/kitraining/Yolox/YOLOX-main/pretrained/${trainingRow.selected_model}`;
|
||||
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
|
||||
} else if (
|
||||
trainingRow.selected_model &&
|
||||
trainingRow.selected_model.toLowerCase() === 'coco' &&
|
||||
(!trainingRow.transfer_learning || trainingRow.transfer_learning === false)
|
||||
) {
|
||||
// If selected_model is 'coco' and not transfer_learning, add modelArg only
|
||||
modelArg = ` -c /pretrained/${trainingRow.selected_model}`;
|
||||
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16 -o ${modelArg}.pth --cache'`;
|
||||
} else {
|
||||
// Default: no modelArg
|
||||
cmd = `bash -c 'source ${yoloxVenv} && python tools/train.py -f ${expSrc} -d 1 -b 8 --fp16' --cache`;
|
||||
}
|
||||
console.log(cmd)
|
||||
const child = spawn(cmd, { shell: true, cwd: yoloxMainDir });
|
||||
child.stdout.pipe(process.stdout);
|
||||
child.stderr.pipe(process.stderr);
|
||||
|
||||
res.json({ message: 'Training started' });
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: 'Failed to start training', details: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
// Get YOLOX training log
|
||||
router.get('/training-log', async (req, res) => {
|
||||
try {
|
||||
const { project_id, training_id } = req.query;
|
||||
const trainingProject = await TrainingProject.findByPk(project_id);
|
||||
const projectName = trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${project_id}`;
|
||||
const outDir = path.join(__dirname, '..', projectName, String(training_id));
|
||||
const logPath = path.join(outDir, 'training.log');
|
||||
if (!fs.existsSync(logPath)) {
|
||||
return res.status(404).json({ error: 'Log not found' });
|
||||
}
|
||||
const logData = fs.readFileSync(logPath, 'utf8');
|
||||
res.json({ log: logData });
|
||||
} catch (err) {
|
||||
res.status(500).json({ error: 'Failed to fetch log', details: err.message });
|
||||
}
|
||||
});
|
||||
|
||||
router.post('/training-projects', upload.single('project_image'), async (req, res) => {
|
||||
try {
|
||||
const { title, description } = req.body;
|
||||
const classes = JSON.parse(req.body.classes);
|
||||
const project_image = req.file ? req.file.buffer : null;
|
||||
const project_image_type = req.file ? req.file.mimetype : null;
|
||||
await TrainingProject.create({
|
||||
title,
|
||||
description,
|
||||
classes,
|
||||
project_image,
|
||||
project_image_type
|
||||
});
|
||||
res.json({ message: 'Project created!' });
|
||||
} catch (error) {
|
||||
console.error('Error creating project:', error);
|
||||
res.status(500).json({ message: 'Failed to create project', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
router.get('/training-projects', async (req, res) => {
|
||||
try {
|
||||
const projects = await TrainingProject.findAll();
|
||||
// Convert BLOB to base64 data URL for each project
|
||||
const serialized = projects.map(project => {
|
||||
const plain = project.get({ plain: true });
|
||||
if (plain.project_image) {
|
||||
const base64 = Buffer.from(plain.project_image).toString('base64');
|
||||
const mimeType = plain.project_image_type || 'image/png';
|
||||
plain.project_image = `data:${mimeType};base64,${base64}`;
|
||||
}
|
||||
return plain;
|
||||
});
|
||||
res.json(serialized);
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
router.get('/update-status', async (req, res) => {
|
||||
res.json(updateStatus)
|
||||
})
|
||||
|
||||
router.get('/label-studio-projects', async (req, res) => {
|
||||
try {
|
||||
const LabelStudioProject = require('../models/LabelStudioProject.js');
|
||||
const Image = require('../models/Images.js');
|
||||
const Annotation = require('../models/Annotation.js');
|
||||
const labelStudioProjects = await LabelStudioProject.findAll();
|
||||
const projectsWithCounts = await Promise.all(labelStudioProjects.map(async project => {
|
||||
const plain = project.get({ plain: true });
|
||||
// Get all images for this project
|
||||
const images = await Image.findAll({ where: { project_id: plain.project_id } });
|
||||
let annotationCounts = {};
|
||||
if (images.length > 0) {
|
||||
const imageIds = images.map(img => img.image_id);
|
||||
// Get all annotations for these images
|
||||
const annotations = await Annotation.findAll({ where: { image_id: imageIds } });
|
||||
// Count by label
|
||||
for (const ann of annotations) {
|
||||
const label = ann.Label;
|
||||
annotationCounts[label] = (annotationCounts[label] || 0) + 1;
|
||||
}
|
||||
}
|
||||
plain.annotationCounts = annotationCounts;
|
||||
return plain;
|
||||
}));
|
||||
res.json(projectsWithCounts);
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to fetch projects', error: error.message });
|
||||
}
|
||||
})
|
||||
|
||||
|
||||
// POST endpoint to create TrainingProjectDetails with all fields
|
||||
router.post('/training-project-details', async (req, res) => {
|
||||
try {
|
||||
const {
|
||||
project_id,
|
||||
annotation_projects,
|
||||
class_map,
|
||||
description
|
||||
} = req.body;
|
||||
if (!project_id || !annotation_projects) {
|
||||
return res.status(400).json({ message: 'Missing required fields' });
|
||||
}
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const created = await TrainingProjectDetails.create({
|
||||
project_id,
|
||||
annotation_projects,
|
||||
class_map: class_map || null,
|
||||
description: description || null
|
||||
});
|
||||
res.json({ message: 'TrainingProjectDetails created', details: created });
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to create TrainingProjectDetails', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// GET endpoint to fetch all TrainingProjectDetails
|
||||
router.get('/training-project-details', async (req, res) => {
|
||||
try {
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const details = await TrainingProjectDetails.findAll();
|
||||
res.json(details);
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to fetch TrainingProjectDetails', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// PUT endpoint to update class_map and description in TrainingProjectDetails
|
||||
router.put('/training-project-details', async (req, res) => {
|
||||
try {
|
||||
const { project_id, class_map, description } = req.body;
|
||||
if (!project_id || !class_map || !description) {
|
||||
return res.status(400).json({ message: 'Missing required fields' });
|
||||
}
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const details = await TrainingProjectDetails.findOne({ where: { project_id } });
|
||||
if (!details) {
|
||||
return res.status(404).json({ message: 'TrainingProjectDetails not found' });
|
||||
}
|
||||
details.class_map = class_map;
|
||||
details.description = description;
|
||||
await details.save();
|
||||
res.json({ message: 'Class map and description updated', details });
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to update class map or description', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// POST endpoint to receive YOLOX settings and save to DB (handles multipart/form-data)
|
||||
router.post('/yolox-settings', upload.any(), async (req, res) => {
|
||||
try {
|
||||
const settings = req.body;
|
||||
// Debug: Log all received fields and types
|
||||
console.log('--- YOLOX settings received ---');
|
||||
console.log('settings:', settings);
|
||||
if (req.files && req.files.length > 0) {
|
||||
console.log('Files received:', req.files.map(f => ({ fieldname: f.fieldname, originalname: f.originalname, size: f.size })));
|
||||
}
|
||||
// Declare requiredFields once
|
||||
const requiredFields = ['project_details_id', 'exp_name', 'max_epoch', 'depth', 'width', 'activation', 'train', 'valid', 'test', 'selected_model', 'transfer_learning'];
|
||||
// Log types of required fields
|
||||
requiredFields.forEach(field => {
|
||||
console.log(`Field '${field}': value='${settings[field]}', type='${typeof settings[field]}'`);
|
||||
});
|
||||
// Map select_model to selected_model if present
|
||||
if (settings && settings.select_model && !settings.selected_model) {
|
||||
settings.selected_model = settings.select_model;
|
||||
delete settings.select_model;
|
||||
}
|
||||
// Lookup project_details_id from project_id
|
||||
if (!settings.project_id || isNaN(Number(settings.project_id))) {
|
||||
throw new Error('Missing or invalid project_id in request. Cannot assign training to a project.');
|
||||
}
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
let details = await TrainingProjectDetails.findOne({ where: { project_id: settings.project_id } });
|
||||
if (!details) {
|
||||
details = await TrainingProjectDetails.create({
|
||||
project_id: settings.project_id,
|
||||
annotation_projects: [],
|
||||
class_map: null,
|
||||
description: null
|
||||
});
|
||||
}
|
||||
settings.project_details_id = details.id;
|
||||
// Map 'act' from frontend to 'activation' for DB
|
||||
if (settings.act !== undefined) {
|
||||
settings.activation = settings.act;
|
||||
delete settings.act;
|
||||
}
|
||||
// Type conversion for DB compatibility
|
||||
[
|
||||
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr', 'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum', 'print_interval', 'eval_interval', 'test_conf', 'nmsthre', 'multiscale_range', 'degrees', 'translate', 'shear', 'train', 'valid', 'test'
|
||||
].forEach(f => {
|
||||
if (settings[f] !== undefined) settings[f] = Number(settings[f]);
|
||||
});
|
||||
// Improved boolean conversion
|
||||
['ema', 'enable_mixup', 'save_history_ckpt'].forEach(f => {
|
||||
if (settings[f] !== undefined) {
|
||||
if (typeof settings[f] === 'string') {
|
||||
settings[f] = settings[f].toLowerCase() === 'true';
|
||||
} else {
|
||||
settings[f] = Boolean(settings[f]);
|
||||
}
|
||||
}
|
||||
});
|
||||
// Improved array conversion
|
||||
['mosaic_scale', 'mixup_scale', 'scale'].forEach(f => {
|
||||
if (settings[f] && typeof settings[f] === 'string') {
|
||||
settings[f] = settings[f]
|
||||
.split(',')
|
||||
.map(s => Number(s.trim()))
|
||||
.filter(n => !isNaN(n));
|
||||
}
|
||||
});
|
||||
// Trim all string fields
|
||||
Object.keys(settings).forEach(f => {
|
||||
if (typeof settings[f] === 'string') settings[f] = settings[f].trim();
|
||||
});
|
||||
// Set default for transfer_learning if missing
|
||||
if (settings.transfer_learning === undefined) settings.transfer_learning = false;
|
||||
// Convert empty string seed to null
|
||||
if ('seed' in settings && (settings.seed === '' || settings.seed === undefined)) {
|
||||
settings.seed = null;
|
||||
}
|
||||
// Validate required fields for training table
|
||||
for (const field of requiredFields) {
|
||||
if (settings[field] === undefined || settings[field] === null || settings[field] === '') {
|
||||
console.error('Missing required field:', field, 'Value:', settings[field]);
|
||||
throw new Error('Missing required field: ' + field);
|
||||
}
|
||||
}
|
||||
console.log('Received YOLOX settings:', settings);
|
||||
// Handle uploaded model file (ckpt_upload)
|
||||
if (req.files && req.files.length > 0) {
|
||||
const ckptFile = req.files.find(f => f.fieldname === 'ckpt_upload');
|
||||
if (ckptFile) {
|
||||
const uploadDir = path.join(__dirname, '..', 'uploads');
|
||||
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
|
||||
const filename = ckptFile.originalname || `uploaded_model_${settings.project_id}.pth`;
|
||||
const filePath = path.join(uploadDir, filename);
|
||||
fs.writeFileSync(filePath, ckptFile.buffer);
|
||||
settings.model_upload = filePath;
|
||||
}
|
||||
}
|
||||
// Save settings to DB only (no file)
|
||||
const { pushYoloxExpToDb } = require('../services/push-yolox-exp.js');
|
||||
const training = await pushYoloxExpToDb(settings);
|
||||
res.json({ message: 'YOLOX settings saved to DB', training });
|
||||
} catch (error) {
|
||||
console.error('Error in /api/yolox-settings:', error.stack || error);
|
||||
res.status(500).json({ message: 'Failed to save YOLOX settings', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// POST endpoint to receive binary model file and save to disk (not DB)
|
||||
router.post('/yolox-settings/upload', async (req, res) => {
|
||||
try {
|
||||
const projectId = req.query.project_id;
|
||||
if (!projectId) return res.status(400).json({ message: 'Missing project_id in query' });
|
||||
// Save file to disk
|
||||
const uploadDir = path.join(__dirname, '..', 'uploads');
|
||||
if (!fs.existsSync(uploadDir)) fs.mkdirSync(uploadDir);
|
||||
const filename = req.headers['x-upload-filename'] || `uploaded_model_${projectId}.pth`;
|
||||
const filePath = path.join(uploadDir, filename);
|
||||
const chunks = [];
|
||||
req.on('data', chunk => chunks.push(chunk));
|
||||
req.on('end', async () => {
|
||||
const buffer = Buffer.concat(chunks);
|
||||
fs.writeFile(filePath, buffer, async err => {
|
||||
if (err) {
|
||||
console.error('Error saving file:', err);
|
||||
return res.status(500).json({ message: 'Failed to save model file', error: err.message });
|
||||
}
|
||||
// Update latest training row for this project with file path
|
||||
try {
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const Training = require('../models/training.js');
|
||||
// Find details row for this project
|
||||
const details = await TrainingProjectDetails.findOne({ where: { project_id: projectId } });
|
||||
if (!details) return res.status(404).json({ message: 'No TrainingProjectDetails found for project_id' });
|
||||
// Find latest training for this details row
|
||||
const training = await Training.findOne({ where: { project_details_id: details.id }, order: [['createdAt', 'DESC']] });
|
||||
if (!training) return res.status(404).json({ message: 'No training found for project_id' });
|
||||
// Save file path to model_upload field
|
||||
training.model_upload = filePath;
|
||||
await training.save();
|
||||
res.json({ message: 'Model file uploaded and saved to disk', filename, trainingId: training.id });
|
||||
} catch (dbErr) {
|
||||
console.error('Error updating training with file path:', dbErr);
|
||||
res.status(500).json({ message: 'File saved but failed to update training row', error: dbErr.message });
|
||||
}
|
||||
});
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error in /api/yolox-settings/upload:', error.stack || error);
|
||||
res.status(500).json({ message: 'Failed to upload model file', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// GET endpoint to fetch all trainings (optionally filtered by project_id)
|
||||
router.get('/trainings', async (req, res) => {
|
||||
try {
|
||||
const project_id = req.query.project_id;
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const Training = require('../models/training.js');
|
||||
if (project_id) {
|
||||
// Find all details rows for this project
|
||||
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id } });
|
||||
if (!detailsRows || detailsRows.length === 0) return res.json([]);
|
||||
// Get all trainings linked to any details row for this project
|
||||
const detailsIds = detailsRows.map(d => d.id);
|
||||
const trainings = await Training.findAll({ where: { project_details_id: detailsIds } });
|
||||
return res.json(trainings);
|
||||
} else {
|
||||
// Return all trainings if no project_id is specified
|
||||
const trainings = await Training.findAll();
|
||||
return res.json(trainings);
|
||||
}
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to fetch trainings', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// DELETE endpoint to remove a training by id
|
||||
router.delete('/trainings/:id', async (req, res) => {
|
||||
try {
|
||||
const Training = require('../models/training.js');
|
||||
const id = req.params.id;
|
||||
const deleted = await Training.destroy({ where: { id } });
|
||||
if (deleted) {
|
||||
res.json({ message: 'Training deleted' });
|
||||
} else {
|
||||
res.status(404).json({ message: 'Training not found' });
|
||||
}
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to delete training', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
// DELETE endpoint to remove a training project and all related entries
|
||||
router.delete('/training-projects/:id', async (req, res) => {
|
||||
try {
|
||||
const projectId = req.params.id;
|
||||
const TrainingProject = require('../models/TrainingProject.js');
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const Training = require('../models/training.js');
|
||||
// Find details row(s) for this project
|
||||
const detailsRows = await TrainingProjectDetails.findAll({ where: { project_id: projectId } });
|
||||
const detailsIds = detailsRows.map(d => d.id);
|
||||
// Delete all trainings linked to these details
|
||||
if (detailsIds.length > 0) {
|
||||
await Training.destroy({ where: { project_details_id: detailsIds } });
|
||||
await TrainingProjectDetails.destroy({ where: { project_id: projectId } });
|
||||
}
|
||||
// Delete the project itself
|
||||
const deleted = await TrainingProject.destroy({ where: { project_id: projectId } });
|
||||
if (deleted) {
|
||||
res.json({ message: 'Training project and all related entries deleted' });
|
||||
} else {
|
||||
res.status(404).json({ message: 'Training project not found' });
|
||||
}
|
||||
} catch (error) {
|
||||
res.status(500).json({ message: 'Failed to delete training project', error: error.message });
|
||||
}
|
||||
});
|
||||
|
||||
module.exports = router;
|
||||
84
backend/routes/api.py
Normal file → Executable file
84
backend/routes/api.py
Normal file → Executable file
@@ -73,7 +73,7 @@ def generate_yolox_json():
|
||||
|
||||
@api_bp.route('/start-yolox-training', methods=['POST'])
|
||||
def start_yolox_training():
|
||||
"""Generate JSONs, exp.py, and start YOLOX training"""
|
||||
"""Generate JSONs, exp.py, and add training to queue"""
|
||||
try:
|
||||
data = request.get_json()
|
||||
project_id = data.get('project_id')
|
||||
@@ -90,9 +90,11 @@ def start_yolox_training():
|
||||
details_id = training.project_details_id
|
||||
|
||||
# Step 1: Generate COCO JSON files
|
||||
# IMPORTANT: Pass training_id (not details_id) so JSON files are generated
|
||||
# in the same location where exp.py expects them
|
||||
from services.generate_json_yolox import generate_training_json
|
||||
print(f'Generating COCO JSON for training {training_id}...')
|
||||
generate_training_json(details_id)
|
||||
generate_training_json(training_id)
|
||||
|
||||
# Step 2: Generate exp.py
|
||||
from services.generate_yolox_exp import save_yolox_exp
|
||||
@@ -113,8 +115,8 @@ def start_yolox_training():
|
||||
print(f'Generating exp.py at {exp_file_path}...')
|
||||
save_yolox_exp(training_id, exp_file_path)
|
||||
|
||||
# Step 3: Start training
|
||||
print(f'Starting YOLOX training for training {training_id}...')
|
||||
# Step 3: Build training command
|
||||
print(f'Preparing training command for training {training_id}...')
|
||||
|
||||
# Get YOLOX configuration from settings
|
||||
yolox_main_dir = get_setting('yolox_path', '/home/kitraining/Yolox/YOLOX-main')
|
||||
@@ -130,11 +132,17 @@ def start_yolox_training():
|
||||
if (training.transfer_learning and
|
||||
isinstance(training.transfer_learning, str) and
|
||||
training.transfer_learning.lower() == 'coco'):
|
||||
model_arg = f'-c {yolox_main_dir}/pretrained/{training.selected_model}.pth'
|
||||
# Use yolox_path setting to construct pretrained model path
|
||||
model_path = os.path.join(yolox_main_dir, 'pretrained', f'{training.selected_model}.pth')
|
||||
model_path = model_path.replace('\\', '/') # Use forward slashes for command line
|
||||
model_arg = f'-c {model_path}'
|
||||
elif (training.selected_model and
|
||||
training.selected_model.lower() == 'coco' and
|
||||
(not training.transfer_learning or training.transfer_learning == False)):
|
||||
model_arg = f'-c {yolox_main_dir}/pretrained/{training.selected_model}.pth'
|
||||
# Use yolox_path setting to construct pretrained model path
|
||||
model_path = os.path.join(yolox_main_dir, 'pretrained', f'{training.selected_model}.pth')
|
||||
model_path = model_path.replace('\\', '/') # Use forward slashes for command line
|
||||
model_arg = f'-c {model_path}'
|
||||
|
||||
# Build base training arguments
|
||||
train_args = f'-f {exp_file_path} -d 1 -b 8 --fp16 --cache'
|
||||
@@ -151,16 +159,18 @@ def start_yolox_training():
|
||||
venv_activate = yolox_venv
|
||||
cmd = f'cmd /c ""{venv_activate}" && python tools\\train.py {train_args}"'
|
||||
else:
|
||||
# Linux: Use bash with source
|
||||
cmd = f'bash -c "source {yolox_venv} && python tools/train.py {train_args}"'
|
||||
# Linux: Use bash with source to activate the venv
|
||||
venv_activate = os.path.join(yolox_venv, 'bin', 'activate')
|
||||
cmd = f'bash -c "source {venv_activate} && python tools/train.py {train_args}"'
|
||||
|
||||
print(f'Training command: {cmd}')
|
||||
|
||||
# Start training in background
|
||||
subprocess.Popen(cmd, shell=True, cwd=yolox_main_dir)
|
||||
# Step 4: Add to training queue
|
||||
from services.training_queue import training_queue
|
||||
training_queue.add_to_queue(training_id, cmd, yolox_main_dir)
|
||||
|
||||
return jsonify({
|
||||
'message': f'JSONs and exp.py generated, training started for training {training_id}',
|
||||
'message': f'Training {training_id} added to queue',
|
||||
'exp_path': exp_file_path
|
||||
})
|
||||
|
||||
@@ -170,6 +180,18 @@ def start_yolox_training():
|
||||
traceback.print_exc()
|
||||
return jsonify({'message': 'Failed to start training', 'error': str(err)}), 500
|
||||
|
||||
@api_bp.route('/training-status', methods=['GET'])
|
||||
def get_training_status():
|
||||
"""Get current training queue status"""
|
||||
try:
|
||||
from services.training_queue import training_queue
|
||||
status = training_queue.get_status()
|
||||
return jsonify(status)
|
||||
|
||||
except Exception as err:
|
||||
print(f'Error getting training status: {err}')
|
||||
return jsonify({'current': None, 'queue': []}), 500
|
||||
|
||||
@api_bp.route('/training-log', methods=['GET'])
|
||||
def training_log():
|
||||
"""Get YOLOX training log"""
|
||||
@@ -477,17 +499,26 @@ def yolox_settings():
|
||||
settings['activation'] = settings['act']
|
||||
del settings['act']
|
||||
|
||||
# Type conversions
|
||||
numeric_fields = [
|
||||
'max_epoch', 'depth', 'width', 'warmup_epochs', 'warmup_lr',
|
||||
'no_aug_epochs', 'min_lr_ratio', 'weight_decay', 'momentum',
|
||||
'print_interval', 'eval_interval', 'test_conf', 'nmsthre',
|
||||
'multiscale_range', 'degrees', 'translate', 'shear',
|
||||
'train', 'valid', 'test'
|
||||
# Type conversions - Integer fields
|
||||
integer_fields = [
|
||||
'max_epoch', 'warmup_epochs', 'no_aug_epochs', 'print_interval',
|
||||
'eval_interval', 'multiscale_range', 'data_num_workers', 'num_classes'
|
||||
]
|
||||
|
||||
for field in numeric_fields:
|
||||
if field in settings:
|
||||
for field in integer_fields:
|
||||
if field in settings and settings[field] not in [None, '']:
|
||||
settings[field] = int(float(settings[field])) # Convert via float to handle "10.0" strings
|
||||
|
||||
# Type conversions - Float fields
|
||||
float_fields = [
|
||||
'depth', 'width', 'warmup_lr', 'min_lr_ratio', 'weight_decay',
|
||||
'momentum', 'test_conf', 'nmsthre', 'degrees', 'translate',
|
||||
'shear', 'mosaic_prob', 'mixup_prob', 'hsv_prob', 'flip_prob',
|
||||
'basic_lr_per_img', 'train', 'valid', 'test'
|
||||
]
|
||||
|
||||
for field in float_fields:
|
||||
if field in settings and settings[field] not in [None, '']:
|
||||
settings[field] = float(settings[field])
|
||||
|
||||
# Boolean conversions
|
||||
@@ -617,6 +648,19 @@ def get_trainings():
|
||||
except Exception as error:
|
||||
return jsonify({'message': 'Failed to fetch trainings', 'error': str(error)}), 500
|
||||
|
||||
@api_bp.route('/trainings/<int:id>', methods=['GET'])
|
||||
def get_training(id):
|
||||
"""Get a single training by id"""
|
||||
try:
|
||||
training = Training.query.get(id)
|
||||
if training:
|
||||
return jsonify(training.to_dict())
|
||||
else:
|
||||
return jsonify({'message': 'Training not found'}), 404
|
||||
|
||||
except Exception as error:
|
||||
return jsonify({'message': 'Failed to fetch training', 'error': str(error)}), 500
|
||||
|
||||
@api_bp.route('/trainings/<int:id>', methods=['DELETE'])
|
||||
def delete_training(id):
|
||||
"""Delete a training by id"""
|
||||
|
||||
0
backend/server.js
Normal file → Executable file
0
backend/server.js
Normal file → Executable file
0
backend/services/__init__.py
Normal file → Executable file
0
backend/services/__init__.py
Normal file → Executable file
@@ -1,92 +0,0 @@
|
||||
const API_URL = 'http://192.168.1.19:8080/api';
|
||||
const API_TOKEN = 'c1cef980b7c73004f4ee880a42839313b863869f';
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
const fetch = require('node-fetch');
|
||||
|
||||
async function fetchLableStudioProject(projectid) {
|
||||
// 1. Trigger export
|
||||
const exportUrl = `${API_URL}/projects/${projectid}/export?exportType=JSON_MIN`;
|
||||
const headers = { Authorization: `Token ${API_TOKEN}` };
|
||||
let res = await fetch(exportUrl, { headers });
|
||||
if (!res.ok) {
|
||||
let errorText = await res.text().catch(() => '');
|
||||
console.error(`Failed to trigger export: ${res.status} ${res.statusText} - ${errorText}`);
|
||||
throw new Error(`Failed to trigger export: ${res.status} ${res.statusText}`);
|
||||
}
|
||||
let data = await res.json();
|
||||
// If data is an array, it's ready
|
||||
if (Array.isArray(data)) return data;
|
||||
// If not, poll for the export file
|
||||
let fileUrl = data.download_url || data.url || null;
|
||||
let tries = 0;
|
||||
while (!fileUrl && tries < 20) {
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
res = await fetch(exportUrl, { headers });
|
||||
if (!res.ok) {
|
||||
let errorText = await res.text().catch(() => '');
|
||||
console.error(`Failed to poll export: ${res.status} ${res.statusText} - ${errorText}`);
|
||||
throw new Error(`Failed to poll export: ${res.status} ${res.statusText}`);
|
||||
}
|
||||
data = await res.json();
|
||||
fileUrl = data.download_url || data.url || null;
|
||||
tries++;
|
||||
}
|
||||
if (!fileUrl) throw new Error('Label Studio export did not become ready');
|
||||
// 2. Download the export file
|
||||
res = await fetch(fileUrl.startsWith('http') ? fileUrl : `${API_URL.replace('/api','')}${fileUrl}`, { headers });
|
||||
if (!res.ok) {
|
||||
let errorText = await res.text().catch(() => '');
|
||||
console.error(`Failed to download export: ${res.status} ${res.statusText} - ${errorText}`);
|
||||
throw new Error(`Failed to download export: ${res.status} ${res.statusText}`);
|
||||
}
|
||||
return await res.json();
|
||||
}
|
||||
|
||||
|
||||
|
||||
async function fetchProjectIdsAndTitles() {
|
||||
try {
|
||||
const response = await fetch(`${API_URL}/projects/`, {
|
||||
headers: {
|
||||
'Authorization': `Token ${API_TOKEN}`,
|
||||
'Content-Type': 'application/json'
|
||||
}
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
let errorText = await response.text().catch(() => '');
|
||||
console.error(`Failed to fetch projects: ${response.status} ${response.statusText} - ${errorText}`);
|
||||
throw new Error(`HTTP error! status: ${response.status}`);
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
if (!data.results || !Array.isArray(data.results)) {
|
||||
throw new Error('API response does not contain results array');
|
||||
}
|
||||
|
||||
// Extract id and title from each project
|
||||
const projects = data.results.map(project => ({
|
||||
id: project.id,
|
||||
title: project.title
|
||||
}));
|
||||
console.log(projects)
|
||||
return projects;
|
||||
|
||||
} catch (error) {
|
||||
console.error('Failed to fetch projects:', error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { fetchLableStudioProject, fetchProjectIdsAndTitles };
|
||||
|
||||
|
||||
|
||||
//getLableStudioProject(20)
|
||||
//fetchProjectIdsAndTitles()
|
||||
0
backend/services/fetch_labelstudio.py
Normal file → Executable file
0
backend/services/fetch_labelstudio.py
Normal file → Executable file
@@ -1,176 +0,0 @@
|
||||
const TrainingProject = require('../models/TrainingProject.js');
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js')
|
||||
const LabelStudioProject = require('../models/LabelStudioProject.js')
|
||||
const Annotation = require('../models/Annotation.js')
|
||||
const Images = require('../models/Images.js')
|
||||
const fs = require('fs');
|
||||
|
||||
|
||||
async function generateTrainingJson(trainingId){
|
||||
// trainingId is now project_details_id
|
||||
const trainingProjectDetails = await TrainingProjectDetails.findByPk(trainingId);
|
||||
if (!trainingProjectDetails) throw new Error('No TrainingProjectDetails found for project_details_id ' + trainingId);
|
||||
const detailsObj = trainingProjectDetails.get({ plain: true });
|
||||
// Get parent project for name
|
||||
const trainingProject = await TrainingProject.findByPk(detailsObj.project_id);
|
||||
// Get split percentages (assume they are stored as train_percent, valid_percent, test_percent)
|
||||
const trainPercent = detailsObj.train_percent || 85;
|
||||
const validPercent = detailsObj.valid_percent || 10;
|
||||
const testPercent = detailsObj.test_percent || 5;
|
||||
|
||||
let cocoImages = [];
|
||||
let cocoAnnotations = [];
|
||||
let cocoCategories = [];
|
||||
let categoryMap = {};
|
||||
let categoryId = 0;
|
||||
let imageid = 0;
|
||||
let annotationid = 0;
|
||||
|
||||
for (const cls of detailsObj.class_map) {
|
||||
const asgMap = [];
|
||||
const listAsg = cls[1];
|
||||
for(const asg of listAsg){
|
||||
asgMap.push ({ original: asg[0], mapped: asg[1] });
|
||||
// Build category list and mapping
|
||||
if (asg[1] && !(asg[1] in categoryMap)) {
|
||||
categoryMap[asg[1]] = categoryId;
|
||||
cocoCategories.push({ id: categoryId, name: asg[1], supercategory: '' });
|
||||
categoryId++;
|
||||
}
|
||||
}
|
||||
const images = await Images.findAll({ where: { project_id: cls[0] } });
|
||||
for(const image of images){
|
||||
imageid += 1;
|
||||
let fileName = image.image_path;
|
||||
if (fileName.includes('%20')) {
|
||||
fileName = fileName.replace(/%20/g, ' ');
|
||||
}
|
||||
if (fileName && fileName.startsWith('/data/local-files/?d=')) {
|
||||
fileName = fileName.replace('/data/local-files/?d=', '');
|
||||
fileName = fileName.replace('/home/kitraining/home/kitraining/', '');
|
||||
}
|
||||
if (fileName && fileName.startsWith('home/kitraining/To_Annotate/')) {
|
||||
fileName = fileName.replace('home/kitraining/To_Annotate/','');
|
||||
}
|
||||
// Get annotations for this image
|
||||
const annotations = await Annotation.findAll({ where: { image_id: image.image_id } });
|
||||
// Use image.width and image.height from DB (populated from original_width/original_height)
|
||||
cocoImages.push({
|
||||
id: imageid,
|
||||
file_name: fileName,
|
||||
width: image.width || 0,
|
||||
height: image.height || 0
|
||||
});
|
||||
for (const annotation of annotations) {
|
||||
// Translate class name using asgMap
|
||||
let mappedClass = annotation.Label;
|
||||
for (const mapEntry of asgMap) {
|
||||
if (annotation.Label === mapEntry.original) {
|
||||
mappedClass = mapEntry.mapped;
|
||||
break;
|
||||
}
|
||||
}
|
||||
// Only add annotation if mappedClass is valid
|
||||
if (mappedClass && mappedClass in categoryMap) {
|
||||
annotationid += 1;
|
||||
let area = 0;
|
||||
if (annotation.width && annotation.height) {
|
||||
area = annotation.width * annotation.height;
|
||||
}
|
||||
cocoAnnotations.push({
|
||||
id: annotationid,
|
||||
image_id: imageid,
|
||||
category_id: categoryMap[mappedClass],
|
||||
bbox: [annotation.x, annotation.y, annotation.width, annotation.height],
|
||||
area: area,
|
||||
iscrowd: annotation.iscrowd || 0
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Shuffle images for random split using seed
|
||||
function seededRandom(seed) {
|
||||
let x = Math.sin(seed++) * 10000;
|
||||
return x - Math.floor(x);
|
||||
}
|
||||
function shuffle(array, seed) {
|
||||
for (let i = array.length - 1; i > 0; i--) {
|
||||
const j = Math.floor(seededRandom(seed + i) * (i + 1));
|
||||
[array[i], array[j]] = [array[j], array[i]];
|
||||
}
|
||||
}
|
||||
// Use seed from detailsObj if present, else default to 42
|
||||
const splitSeed = detailsObj.seed !== undefined && detailsObj.seed !== null ? Number(detailsObj.seed) : 42;
|
||||
shuffle(cocoImages, splitSeed);
|
||||
|
||||
// Split images
|
||||
const totalImages = cocoImages.length;
|
||||
const trainCount = Math.floor(totalImages * trainPercent / 100);
|
||||
const validCount = Math.floor(totalImages * validPercent / 100);
|
||||
const testCount = totalImages - trainCount - validCount;
|
||||
|
||||
const trainImages = cocoImages.slice(0, trainCount);
|
||||
const validImages = cocoImages.slice(trainCount, trainCount + validCount);
|
||||
const testImages = cocoImages.slice(trainCount + validCount);
|
||||
|
||||
// Helper to get image ids for each split
|
||||
const trainImageIds = new Set(trainImages.map(img => img.id));
|
||||
const validImageIds = new Set(validImages.map(img => img.id));
|
||||
const testImageIds = new Set(testImages.map(img => img.id));
|
||||
|
||||
// Split annotations
|
||||
const trainAnnotations = cocoAnnotations.filter(ann => trainImageIds.has(ann.image_id));
|
||||
const validAnnotations = cocoAnnotations.filter(ann => validImageIds.has(ann.image_id));
|
||||
const testAnnotations = cocoAnnotations.filter(ann => testImageIds.has(ann.image_id));
|
||||
|
||||
// Build final COCO JSONs with info section
|
||||
const buildCocoJson = (images, annotations, categories) => ({
|
||||
images,
|
||||
annotations,
|
||||
categories
|
||||
});
|
||||
|
||||
// Build COCO JSONs with info section
|
||||
const trainJson = buildCocoJson(trainImages, trainAnnotations, cocoCategories);
|
||||
const validJson = buildCocoJson(validImages, validAnnotations, cocoCategories);
|
||||
const testJson = buildCocoJson(testImages, testAnnotations, cocoCategories);
|
||||
|
||||
// Create output directory: projectname/trainingid/annotations
|
||||
const projectName = trainingProject && trainingProject.name ? trainingProject.name.replace(/\s+/g, '_') : `project_${detailsObj.project_id}`;
|
||||
const outDir = `${projectName}/${trainingId}`;
|
||||
const annotationsDir = `/home/kitraining/To_Annotate/annotations`;
|
||||
if (!fs.existsSync(annotationsDir)) {
|
||||
fs.mkdirSync(annotationsDir, { recursive: true });
|
||||
}
|
||||
|
||||
// Write to files in the annotations directory
|
||||
const trainPath = `${annotationsDir}/coco_project_${trainingId}_train.json`;
|
||||
const validPath = `${annotationsDir}/coco_project_${trainingId}_valid.json`;
|
||||
const testPath = `${annotationsDir}/coco_project_${trainingId}_test.json`;
|
||||
fs.writeFileSync(trainPath, JSON.stringify(trainJson, null, 2));
|
||||
fs.writeFileSync(validPath, JSON.stringify(validJson, null, 2));
|
||||
fs.writeFileSync(testPath, JSON.stringify(testJson, null, 2));
|
||||
console.log(`COCO JSON splits written to ${annotationsDir} for trainingId ${trainingId}`);
|
||||
|
||||
|
||||
|
||||
// Also generate inference exp.py in the same output directory as exp.py (project folder in workspace)
|
||||
const { generateYoloxInferenceExp } = require('./generate-yolox-exp');
|
||||
const path = require('path');
|
||||
const projectFolder = path.join(__dirname, '..', projectName, String(trainingId));
|
||||
if (!fs.existsSync(projectFolder)) {
|
||||
fs.mkdirSync(projectFolder, { recursive: true });
|
||||
}
|
||||
const inferenceExpPath = path.join(projectFolder, 'exp_infer.py');
|
||||
generateYoloxInferenceExp(trainingId).then(expContent => {
|
||||
fs.writeFileSync(inferenceExpPath, expContent);
|
||||
console.log(`Inference exp.py written to ${inferenceExpPath}`);
|
||||
}).catch(err => {
|
||||
console.error('Failed to generate inference exp.py:', err);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
module.exports = {generateTrainingJson};
|
||||
@@ -1,135 +0,0 @@
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const Training = require('../models/training.js');
|
||||
const TrainingProject = require('../models/TrainingProject.js');
|
||||
|
||||
// Remove Python comments and legacy code
|
||||
const exp_names = [
|
||||
'YOLOX-s',
|
||||
'YOLOX-m',
|
||||
'YOLOX-l',
|
||||
'YOLOX-x',
|
||||
'YOLOX-Darknet53', //todo
|
||||
'YOLOX-Nano',
|
||||
'YOLOX-Tiny'
|
||||
]
|
||||
|
||||
//TODO: Clean up generation of exp_names.py and remove second exp creation!!!
|
||||
|
||||
|
||||
// Refactored: Accept trainingId, fetch info from DB
|
||||
async function generateYoloxExp(trainingId) {
|
||||
// Fetch training row from DB by project_details_id if not found by PK
|
||||
let training = await Training.findByPk(trainingId);
|
||||
if (!training) {
|
||||
training = await Training.findOne({ where: { project_details_id: trainingId } });
|
||||
}
|
||||
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
|
||||
|
||||
// If transfer_learning is 'coco', just return the path to the default exp.py
|
||||
if (training.transfer_learning === 'coco') {
|
||||
const selectedModel = training.selected_model.toLowerCase().replace('-', '_');
|
||||
const expSourcePath = `/home/kitraining/Yolox/YOLOX-main/exps/default/${selectedModel}.py`;
|
||||
if (!fs.existsSync(expSourcePath)) {
|
||||
throw new Error(`Default exp.py not found for model: ${selectedModel} at ${expSourcePath}`);
|
||||
}
|
||||
// Copy to project folder (e.g., /home/kitraining/coco_tool/backend/project_XX/YY/exp.py)
|
||||
const projectDetailsId = training.project_details_id;
|
||||
const projectFolder = path.resolve(__dirname, `../project_23/${projectDetailsId}`);
|
||||
if (!fs.existsSync(projectFolder)) {
|
||||
fs.mkdirSync(projectFolder, { recursive: true });
|
||||
}
|
||||
const expDestPath = path.join(projectFolder, 'exp.py');
|
||||
fs.copyFileSync(expSourcePath, expDestPath);
|
||||
return { type: 'default', expPath: expDestPath };
|
||||
}
|
||||
|
||||
// If transfer_learning is 'sketch', generate a custom exp.py as before
|
||||
if (training.transfer_learning === 'sketch') {
|
||||
// ...existing custom exp.py generation logic here (copy from previous implementation)...
|
||||
// For brevity, you can call generateYoloxInferenceExp or similar here, or inline the logic.
|
||||
// Example:
|
||||
const expContent = await generateYoloxInferenceExp(trainingId);
|
||||
return { type: 'custom', expContent };
|
||||
}
|
||||
|
||||
throw new Error('Unknown transfer_learning type: ' + training.transfer_learning);
|
||||
}
|
||||
|
||||
async function saveYoloxExp(trainingId, outPath) {
|
||||
const expResult = await generateYoloxExp(trainingId);
|
||||
if (expResult.type === 'custom' && expResult.expContent) {
|
||||
fs.writeFileSync(outPath, expResult.expContent);
|
||||
return outPath;
|
||||
} else if (expResult.type === 'default' && expResult.expPath) {
|
||||
// Optionally copy the file if outPath is different
|
||||
if (expResult.expPath !== outPath) {
|
||||
fs.copyFileSync(expResult.expPath, outPath);
|
||||
}
|
||||
return outPath;
|
||||
} else {
|
||||
throw new Error('Unknown expResult type or missing content');
|
||||
}
|
||||
}
|
||||
|
||||
// Generate a second exp.py for inference, using the provided template and DB values
|
||||
async function generateYoloxInferenceExp(trainingId, options = {}) {
|
||||
let training = await Training.findByPk(trainingId);
|
||||
if (!training) {
|
||||
training = await Training.findOne({ where: { project_details_id: trainingId } });
|
||||
}
|
||||
if (!training) throw new Error('Training not found for trainingId or project_details_id: ' + trainingId);
|
||||
// Always use the trainingId (project_details_id) for annotation file names
|
||||
const projectDetailsId = training.project_details_id;
|
||||
const dataDir = options.data_dir || '/home/kitraining/To_Annotate/';
|
||||
const trainAnn = options.train_ann || `coco_project_${trainingId}_train.json`;
|
||||
const valAnn = options.val_ann || `coco_project_${trainingId}_valid.json`;
|
||||
const testAnn = options.test_ann || `coco_project_${trainingId}_test.json`;
|
||||
// Get num_classes from TrainingProject.classes JSON
|
||||
let numClasses = 80;
|
||||
try {
|
||||
const trainingProject = await TrainingProject.findByPk(projectDetailsId);
|
||||
if (trainingProject && trainingProject.classes) {
|
||||
let classesArr = trainingProject.classes;
|
||||
if (typeof classesArr === 'string') {
|
||||
classesArr = JSON.parse(classesArr);
|
||||
}
|
||||
if (Array.isArray(classesArr)) {
|
||||
numClasses = classesArr.filter(c => c !== null && c !== undefined && c !== '').length;
|
||||
} else if (typeof classesArr === 'object' && classesArr !== null) {
|
||||
numClasses = Object.keys(classesArr).filter(k => classesArr[k] !== null && classesArr[k] !== undefined && classesArr[k] !== '').length;
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('Could not determine num_classes from TrainingProject.classes:', e);
|
||||
}
|
||||
const depth = options.depth || training.depth || 1.00;
|
||||
const width = options.width || training.width || 1.00;
|
||||
const inputSize = options.input_size || training.input_size || [640, 640];
|
||||
const mosaicScale = options.mosaic_scale || training.mosaic_scale || [0.1, 2];
|
||||
const randomSize = options.random_size || training.random_size || [10, 20];
|
||||
const testSize = options.test_size || training.test_size || [640, 640];
|
||||
const expName = options.exp_name || 'inference_exp';
|
||||
const enableMixup = options.enable_mixup !== undefined ? options.enable_mixup : false;
|
||||
let expContent = '';
|
||||
expContent += `#!/usr/bin/env python3\n# -*- coding:utf-8 -*-\n# Copyright (c) Megvii, Inc. and its affiliates.\n\nimport os\n\nfrom yolox.exp import Exp as MyExp\n\n\nclass Exp(MyExp):\n def __init__(self):\n super(Exp, self).__init__()\n self.data_dir = "${dataDir}"\n self.train_ann = "${trainAnn}"\n self.val_ann = "${valAnn}"\n self.test_ann = "coco_project_${trainingId}_test.json"\n self.num_classes = ${numClasses}\n`;
|
||||
// Set pretrained_ckpt if transfer_learning is 'coco'
|
||||
if (training.transfer_learning && typeof training.transfer_learning === 'string' && training.transfer_learning.toLowerCase() === 'coco') {
|
||||
const yoloxBaseDir = '/home/kitraining/Yolox/YOLOX-main';
|
||||
const selectedModel = training.selected_model ? training.selected_model.replace(/\.pth$/i, '') : '';
|
||||
if (selectedModel) {
|
||||
expContent += ` self.pretrained_ckpt = r'${yoloxBaseDir}/pretrained/${selectedModel}.pth'\n`;
|
||||
}
|
||||
}
|
||||
expContent += ` self.depth = ${depth}\n self.width = ${width}\n self.input_size = (${Array.isArray(inputSize) ? inputSize.join(', ') : inputSize})\n self.mosaic_scale = (${Array.isArray(mosaicScale) ? mosaicScale.join(', ') : mosaicScale})\n self.random_size = (${Array.isArray(randomSize) ? randomSize.join(', ') : randomSize})\n self.test_size = (${Array.isArray(testSize) ? testSize.join(', ') : testSize})\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(".")[0]\n self.enable_mixup = ${enableMixup ? 'True' : 'False'}\n`;
|
||||
return expContent;
|
||||
}
|
||||
|
||||
// Save inference exp.py to a custom path
|
||||
async function saveYoloxInferenceExp(trainingId, outPath, options = {}) {
|
||||
const expContent = await generateYoloxInferenceExp(trainingId, options);
|
||||
fs.writeFileSync(outPath, expContent);
|
||||
return outPath;
|
||||
}
|
||||
|
||||
module.exports = { generateYoloxExp, saveYoloxExp, generateYoloxInferenceExp, saveYoloxInferenceExp };
|
||||
81
backend/services/generate_json_yolox.py
Normal file → Executable file
81
backend/services/generate_json_yolox.py
Normal file → Executable file
@@ -7,12 +7,30 @@ from models.Images import Image
|
||||
from models.Annotation import Annotation
|
||||
|
||||
def generate_training_json(training_id):
|
||||
"""Generate COCO JSON for training, validation, and test sets"""
|
||||
# training_id is now project_details_id
|
||||
training_project_details = TrainingProjectDetails.query.get(training_id)
|
||||
"""Generate COCO JSON for training, validation, and test sets
|
||||
|
||||
Args:
|
||||
training_id: Can be either a Training.id or TrainingProjectDetails.id
|
||||
Function will automatically detect which one and find the correct details_id
|
||||
"""
|
||||
from models.training import Training
|
||||
|
||||
# First, try to get as a Training record
|
||||
training_record = Training.query.get(training_id)
|
||||
|
||||
if training_record:
|
||||
# It's a Training.id - use its project_details_id
|
||||
details_id = training_record.project_details_id
|
||||
print(f'[generate_training_json] Using training_id={training_id}, mapped to project_details_id={details_id}')
|
||||
else:
|
||||
# Try as TrainingProjectDetails.id directly
|
||||
details_id = training_id
|
||||
print(f'[generate_training_json] Using training_id={training_id} as project_details_id directly')
|
||||
|
||||
training_project_details = TrainingProjectDetails.query.get(details_id)
|
||||
|
||||
if not training_project_details:
|
||||
raise Exception(f'No TrainingProjectDetails found for project_details_id {training_id}')
|
||||
raise Exception(f'No TrainingProjectDetails found for id {training_id} (details_id: {details_id})')
|
||||
|
||||
details_obj = training_project_details.to_dict()
|
||||
|
||||
@@ -110,22 +128,35 @@ def generate_training_json(training_id):
|
||||
break
|
||||
|
||||
# Construct ABSOLUTE path using data_dir
|
||||
# Normalize data_dir - ensure it uses backslashes for Windows
|
||||
# Detect platform for proper path handling
|
||||
import platform
|
||||
is_windows = platform.system() == 'Windows'
|
||||
|
||||
# Normalize data_dir and file_name based on platform
|
||||
if is_windows:
|
||||
# Windows: use backslashes
|
||||
normalized_data_dir = data_dir.rstrip('/\\').replace('/', '\\')
|
||||
file_name = file_name.replace('/', '\\')
|
||||
else:
|
||||
# Linux/Mac: use forward slashes
|
||||
normalized_data_dir = data_dir.rstrip('/\\').replace('\\', '/')
|
||||
file_name = file_name.replace('\\', '/')
|
||||
|
||||
# Check if already absolute path
|
||||
if not (file_name.startswith('\\\\') or (len(file_name) > 1 and file_name[1] == ':')):
|
||||
is_absolute = False
|
||||
if is_windows:
|
||||
is_absolute = file_name.startswith('\\\\') or (len(file_name) > 1 and file_name[1] == ':')
|
||||
else:
|
||||
is_absolute = file_name.startswith('/')
|
||||
|
||||
if not is_absolute:
|
||||
# It's a relative path, combine with data_dir
|
||||
# For UNC paths, we need to manually concatenate to preserve \\
|
||||
if normalized_data_dir.startswith('\\\\'):
|
||||
# UNC path
|
||||
file_name = normalized_data_dir + '\\' + file_name.replace('/', '\\')
|
||||
if is_windows and normalized_data_dir.startswith('\\\\'):
|
||||
# Windows UNC path
|
||||
file_name = normalized_data_dir + '\\' + file_name
|
||||
else:
|
||||
# Regular path
|
||||
file_name = os.path.join(normalized_data_dir, file_name.replace('/', '\\'))
|
||||
else:
|
||||
# Already absolute, just normalize separators
|
||||
file_name = file_name.replace('/', '\\')
|
||||
# Regular path (Windows or Linux)
|
||||
file_name = os.path.join(normalized_data_dir, file_name)
|
||||
|
||||
# Get annotations for this image
|
||||
annotations = Annotation.query.filter_by(image_id=image.image_id).all()
|
||||
@@ -218,13 +249,19 @@ def generate_training_json(training_id):
|
||||
|
||||
project_name = training_project.title.replace(' ', '_') if training_project and training_project.title else f'project_{details_obj["project_id"]}'
|
||||
|
||||
# Get training record to use its name for folder
|
||||
training_record = Training.query.filter_by(project_details_id=training_id).first()
|
||||
training_folder_name = f"{training_record.exp_name or training_record.training_name or 'training'}_{training_record.id}" if training_record else str(training_id)
|
||||
training_folder_name = training_folder_name.replace(' ', '_')
|
||||
# Get training record to use its name and ID for folder and file names
|
||||
# Use the same training_id that was passed in (if it was a Training.id)
|
||||
# or find the first training for this details_id
|
||||
if not training_record:
|
||||
training_record = Training.query.filter_by(project_details_id=details_id).first()
|
||||
|
||||
# Use training_record.id for file names to match what generate_yolox_exp expects
|
||||
training_file_id = training_record.id if training_record else training_id
|
||||
if training_record:
|
||||
training_folder_name = f"{training_record.exp_name or training_record.training_name or 'training'}_{training_record.id}"
|
||||
training_folder_name = training_folder_name.replace(' ', '_')
|
||||
training_file_id = training_record.id
|
||||
else:
|
||||
training_folder_name = str(details_id)
|
||||
training_file_id = details_id
|
||||
|
||||
# Save annotations to the configured output folder
|
||||
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
|
||||
@@ -242,7 +279,7 @@ def generate_training_json(training_id):
|
||||
with open(test_path, 'w') as f:
|
||||
json.dump(test_json, f, indent=2)
|
||||
|
||||
print(f'COCO JSON splits written to {annotations_dir} for trainingId {training_id}')
|
||||
print(f'COCO JSON splits written to {annotations_dir} for training_id={training_file_id} (details_id={details_id})')
|
||||
|
||||
# Also generate inference exp.py
|
||||
from services.generate_yolox_exp import generate_yolox_inference_exp
|
||||
|
||||
68
backend/services/generate_yolox_exp.py
Normal file → Executable file
68
backend/services/generate_yolox_exp.py
Normal file → Executable file
@@ -220,6 +220,10 @@ def generate_yolox_inference_exp(training_id, options=None, use_base_config=Fals
|
||||
annotations_parent_dir = os.path.join(output_base_path, project_name, training_folder_name)
|
||||
annotations_parent_escaped = annotations_parent_dir.replace('\\', '\\\\')
|
||||
|
||||
# Set output directory for checkpoints - models subdirectory
|
||||
models_dir = os.path.join(annotations_parent_dir, 'models')
|
||||
models_dir_escaped = models_dir.replace('\\', '\\\\')
|
||||
|
||||
# Build exp content
|
||||
exp_content = f'''#!/usr/bin/env python3
|
||||
# -*- coding:utf-8 -*-
|
||||
@@ -235,6 +239,7 @@ class Exp(MyExp):
|
||||
super(Exp, self).__init__()
|
||||
self.data_dir = "{data_dir_escaped}" # Where images are located
|
||||
self.annotations_dir = "{annotations_parent_escaped}" # Where annotation JSONs are located
|
||||
self.output_dir = "{models_dir_escaped}" # Where checkpoints will be saved
|
||||
self.train_ann = "{train_ann}"
|
||||
self.val_ann = "{val_ann}"
|
||||
self.test_ann = "{test_ann}"
|
||||
@@ -252,21 +257,46 @@ class Exp(MyExp):
|
||||
if selected_model:
|
||||
exp_content += f" self.pretrained_ckpt = r'{yolox_base_dir}/pretrained/{selected_model}.pth'\n"
|
||||
|
||||
# Format arrays
|
||||
def format_value(val):
|
||||
# Format arrays and values for Python code generation
|
||||
# Integer-only parameters (sizes, epochs, intervals)
|
||||
integer_params = {
|
||||
'input_size', 'test_size', 'random_size', 'max_epoch', 'warmup_epochs',
|
||||
'no_aug_epochs', 'print_interval', 'eval_interval', 'multiscale_range',
|
||||
'data_num_workers', 'num_classes'
|
||||
}
|
||||
|
||||
def format_value(val, param_name=''):
|
||||
if isinstance(val, (list, tuple)):
|
||||
return '(' + ', '.join(map(str, val)) + ')'
|
||||
# Check if this parameter should have integer values
|
||||
if param_name in integer_params:
|
||||
# Convert all values to integers
|
||||
formatted_items = [str(int(float(item))) if isinstance(item, (int, float)) else str(item) for item in val]
|
||||
else:
|
||||
# Keep as floats or original type
|
||||
formatted_items = []
|
||||
for item in val:
|
||||
if isinstance(item, float):
|
||||
formatted_items.append(str(item))
|
||||
elif isinstance(item, int):
|
||||
formatted_items.append(str(item))
|
||||
else:
|
||||
formatted_items.append(str(item))
|
||||
return '(' + ', '.join(formatted_items) + ')'
|
||||
elif isinstance(val, bool):
|
||||
return str(val)
|
||||
elif isinstance(val, str):
|
||||
return f'"{val}"'
|
||||
elif isinstance(val, int):
|
||||
return str(val)
|
||||
elif isinstance(val, float):
|
||||
return str(val)
|
||||
else:
|
||||
return str(val)
|
||||
|
||||
# Add all config parameters to exp
|
||||
for key, value in config.items():
|
||||
if key not in ['exp_name']: # exp_name is handled separately
|
||||
exp_content += f" self.{key} = {format_value(value)}\n"
|
||||
exp_content += f" self.{key} = {format_value(value, key)}\n"
|
||||
|
||||
# Add get_dataset override using name parameter for image directory
|
||||
exp_content += '''
|
||||
@@ -289,7 +319,7 @@ class Exp(MyExp):
|
||||
|
||||
def get_eval_dataset(self, **kwargs):
|
||||
"""Override eval dataset using name parameter"""
|
||||
from yolox.data import COCODataset
|
||||
from yolox.data import COCODataset, ValTransform
|
||||
|
||||
testdev = kwargs.get("testdev", False)
|
||||
legacy = kwargs.get("legacy", False)
|
||||
@@ -299,8 +329,34 @@ class Exp(MyExp):
|
||||
json_file=self.val_ann if not testdev else self.test_ann,
|
||||
name="",
|
||||
img_size=self.test_size,
|
||||
preproc=None, # No preprocessing for evaluation
|
||||
preproc=ValTransform(legacy=legacy), # Use proper validation transform
|
||||
)
|
||||
|
||||
def get_eval_loader(self, batch_size, is_distributed, **kwargs):
|
||||
"""Standard YOLOX eval loader - matches official implementation"""
|
||||
import torch
|
||||
import torch.distributed as dist
|
||||
from torch.utils.data import DataLoader
|
||||
|
||||
valdataset = self.get_eval_dataset(**kwargs)
|
||||
|
||||
if is_distributed:
|
||||
batch_size = batch_size // dist.get_world_size()
|
||||
sampler = torch.utils.data.distributed.DistributedSampler(
|
||||
valdataset, shuffle=False
|
||||
)
|
||||
else:
|
||||
sampler = torch.utils.data.SequentialSampler(valdataset)
|
||||
|
||||
dataloader_kwargs = {
|
||||
"num_workers": self.data_num_workers,
|
||||
"pin_memory": True,
|
||||
"sampler": sampler,
|
||||
}
|
||||
dataloader_kwargs["batch_size"] = batch_size
|
||||
val_loader = DataLoader(valdataset, **dataloader_kwargs)
|
||||
|
||||
return val_loader
|
||||
'''
|
||||
|
||||
# Add exp_name at the end (uses dynamic path)
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
const Training = require('../models/training.js');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
async function pushYoloxExpToDb(settings) {
|
||||
// Normalize boolean and array fields for DB
|
||||
const normalized = { ...settings };
|
||||
// Map 'act' from frontend to 'activation' for DB
|
||||
if (normalized.act !== undefined) {
|
||||
normalized.activation = normalized.act;
|
||||
delete normalized.act;
|
||||
}
|
||||
// Convert 'on'/'off' to boolean for save_history_ckpt
|
||||
if (typeof normalized.save_history_ckpt === 'string') {
|
||||
normalized.save_history_ckpt = normalized.save_history_ckpt === 'on' ? true : false;
|
||||
}
|
||||
// Convert comma-separated strings to arrays for input_size, test_size, mosaic_scale, mixup_scale
|
||||
['input_size', 'test_size', 'mosaic_scale', 'mixup_scale'].forEach(key => {
|
||||
if (typeof normalized[key] === 'string') {
|
||||
const arr = normalized[key].split(',').map(v => parseFloat(v.trim()));
|
||||
normalized[key] = arr.length === 1 ? arr[0] : arr;
|
||||
}
|
||||
});
|
||||
// Find TrainingProjectDetails for this project
|
||||
const TrainingProjectDetails = require('../models/TrainingProjectDetails.js');
|
||||
const details = await TrainingProjectDetails.findOne({ where: { project_id: normalized.project_id } });
|
||||
if (!details) throw new Error('TrainingProjectDetails not found for project_id ' + normalized.project_id);
|
||||
normalized.project_details_id = details.id;
|
||||
// Create DB row
|
||||
const training = await Training.create(normalized);
|
||||
return training;
|
||||
}
|
||||
|
||||
async function generateYoloxExpFromDb(trainingId) {
|
||||
// Fetch training row from DB
|
||||
const training = await Training.findByPk(trainingId);
|
||||
if (!training) throw new Error('Training not found');
|
||||
// Template for exp.py
|
||||
const expTemplate = `#!/usr/bin/env python3\n# Copyright (c) Megvii Inc. All rights reserved.\n\nimport os\nimport random\n\nimport torch\nimport torch.distributed as dist\nimport torch.nn as nn\n\nfrom .base_exp import BaseExp\n\n__all__ = [\"Exp\", \"check_exp_value\"]\n\nclass Exp(BaseExp):\n def __init__(self):\n super().__init__()\n\n # ---------------- model config ---------------- #\n self.num_classes = ${training.num_classes || 80}\n self.depth = ${training.depth || 1.00}\n self.width = ${training.width || 1.00}\n self.act = \"${training.activation || training.act || 'silu'}\"\n\n # ---------------- dataloader config ---------------- #\n self.data_num_workers = ${training.data_num_workers || 4}\n self.input_size = (${Array.isArray(training.input_size) ? training.input_size.join(', ') : '640, 640'})\n self.multiscale_range = ${training.multiscale_range || 5}\n self.data_dir = ${training.data_dir ? `\"${training.data_dir}\"` : 'None'}\n self.train_ann = \"${training.train_ann || 'instances_train2017.json'}\"\n self.val_ann = \"${training.val_ann || 'instances_val2017.json'}\"\n self.test_ann = \"${training.test_ann || 'instances_test2017.json'}\"\n\n # --------------- transform config ----------------- #\n self.mosaic_prob = ${training.mosaic_prob !== undefined ? training.mosaic_prob : 1.0}\n self.mixup_prob = ${training.mixup_prob !== undefined ? training.mixup_prob : 1.0}\n self.hsv_prob = ${training.hsv_prob !== undefined ? training.hsv_prob : 1.0}\n self.flip_prob = ${training.flip_prob !== undefined ? training.flip_prob : 0.5}\n self.degrees = ${training.degrees !== undefined ? training.degrees : 10.0}\n self.translate = ${training.translate !== undefined ? training.translate : 0.1}\n self.mosaic_scale = (${Array.isArray(training.mosaic_scale) ? training.mosaic_scale.join(', ') : '0.1, 2'})\n self.enable_mixup = ${training.enable_mixup !== undefined ? training.enable_mixup : true}\n self.mixup_scale = (${Array.isArray(training.mixup_scale) ? training.mixup_scale.join(', ') : '0.5, 1.5'})\n self.shear = ${training.shear !== undefined ? training.shear : 2.0}\n\n # -------------- training config --------------------- #\n self.warmup_epochs = ${training.warmup_epochs !== undefined ? training.warmup_epochs : 5}\n self.max_epoch = ${training.max_epoch !== undefined ? training.max_epoch : 300}\n self.warmup_lr = ${training.warmup_lr !== undefined ? training.warmup_lr : 0}\n self.min_lr_ratio = ${training.min_lr_ratio !== undefined ? training.min_lr_ratio : 0.05}\n self.basic_lr_per_img = ${training.basic_lr_per_img !== undefined ? training.basic_lr_per_img : 0.01 / 64.0}\n self.scheduler = \"${training.scheduler || 'yoloxwarmcos'}\"\n self.no_aug_epochs = ${training.no_aug_epochs !== undefined ? training.no_aug_epochs : 15}\n self.ema = ${training.ema !== undefined ? training.ema : true}\n self.weight_decay = ${training.weight_decay !== undefined ? training.weight_decay : 5e-4}\n self.momentum = ${training.momentum !== undefined ? training.momentum : 0.9}\n self.print_interval = ${training.print_interval !== undefined ? training.print_interval : 10}\n self.eval_interval = ${training.eval_interval !== undefined ? training.eval_interval : 10}\n self.save_history_ckpt = ${training.save_history_ckpt !== undefined ? training.save_history_ckpt : true}\n self.exp_name = os.path.split(os.path.realpath(__file__))[1].split(\".\")[0]\n\n # ----------------- testing config ------------------ #\n self.test_size = (${Array.isArray(training.test_size) ? training.test_size.join(', ') : '640, 640'})\n self.test_conf = ${training.test_conf !== undefined ? training.test_conf : 0.01}\n self.nmsthre = ${training.nmsthre !== undefined ? training.nmsthre : 0.65}\n\n # ... rest of the template ...\n\ndef check_exp_value(exp: Exp):\n h, w = exp.input_size\n assert h % 32 == 0 and w % 32 == 0, \"input size must be multiples of 32\"\n`;
|
||||
// Save to file in output directory
|
||||
const outDir = path.join(__dirname, '../../', training.project_id ? `project_${training.project_id}/${trainingId}` : 'exp_files');
|
||||
if (!fs.existsSync(outDir)) fs.mkdirSync(outDir, { recursive: true });
|
||||
const filePath = path.join(outDir, 'exp.py');
|
||||
fs.writeFileSync(filePath, expTemplate);
|
||||
return filePath;
|
||||
}
|
||||
|
||||
module.exports = { pushYoloxExpToDb, generateYoloxExpFromDb };
|
||||
0
backend/services/push_yolox_exp.py
Normal file → Executable file
0
backend/services/push_yolox_exp.py
Normal file → Executable file
@@ -1,120 +0,0 @@
|
||||
const sequelize = require('../database/database.js');
|
||||
const { Project, Img, Ann } = require('../models');
|
||||
const { fetchLableStudioProject, fetchProjectIdsAndTitles } = require('./fetch-labelstudio.js');
|
||||
|
||||
const updateStatus = { running: false };
|
||||
|
||||
async function seedLabelStudio() {
|
||||
updateStatus.running = true;
|
||||
console.log('Seeding started');
|
||||
try {
|
||||
await sequelize.sync();
|
||||
const projects = await fetchProjectIdsAndTitles();
|
||||
|
||||
for (const project of projects) {
|
||||
console.log(`Processing project ${project.id} (${project.title})`);
|
||||
|
||||
// Upsert project in DB
|
||||
await Project.upsert({ project_id: project.id, title: project.title });
|
||||
|
||||
// Fetch project data (annotations array)
|
||||
const data = await fetchLableStudioProject(project.id);
|
||||
if (!Array.isArray(data) || data.length === 0) {
|
||||
console.log(`No annotation data for project ${project.id}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Remove old images and annotations for this project
|
||||
const oldImages = await Img.findAll({ where: { project_id: project.id } });
|
||||
const oldImageIds = oldImages.map(img => img.image_id);
|
||||
if (oldImageIds.length > 0) {
|
||||
await Ann.destroy({ where: { image_id: oldImageIds } });
|
||||
await Img.destroy({ where: { project_id: project.id } });
|
||||
console.log(`Deleted ${oldImageIds.length} old images and their annotations for project ${project.id}`);
|
||||
}
|
||||
|
||||
// Prepare arrays
|
||||
const imagesBulk = [];
|
||||
const annsBulk = [];
|
||||
|
||||
for (const ann of data) {
|
||||
// Extract width/height
|
||||
let width = null;
|
||||
let height = null;
|
||||
if (Array.isArray(ann.label_rectangles) && ann.label_rectangles.length > 0) {
|
||||
width = ann.label_rectangles[0].original_width;
|
||||
height = ann.label_rectangles[0].original_height;
|
||||
} else if (Array.isArray(ann.label) && ann.label.length > 0 && ann.label[0].original_width && ann.label[0].original_height) {
|
||||
width = ann.label[0].original_width;
|
||||
height = ann.label[0].original_height;
|
||||
}
|
||||
|
||||
// Only push image and annotations if width and height are valid
|
||||
if (width && height) {
|
||||
imagesBulk.push({
|
||||
project_id: project.id,
|
||||
image_path: ann.image,
|
||||
width,
|
||||
height
|
||||
});
|
||||
|
||||
// Handle multiple annotations per image
|
||||
if (Array.isArray(ann.label_rectangles)) {
|
||||
for (const ann_detail of ann.label_rectangles) {
|
||||
annsBulk.push({
|
||||
image_path: ann.image,
|
||||
x: (ann_detail.x * width) / 100,
|
||||
y: (ann_detail.y * height) / 100,
|
||||
width: (ann_detail.width * width) / 100,
|
||||
height: (ann_detail.height * height) / 100,
|
||||
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
|
||||
});
|
||||
}
|
||||
} else if (Array.isArray(ann.label)) {
|
||||
for (const ann_detail of ann.label) {
|
||||
annsBulk.push({
|
||||
image_path: ann.image,
|
||||
x: (ann_detail.x * width) / 100,
|
||||
y: (ann_detail.y * height) / 100,
|
||||
width: (ann_detail.width * width) / 100,
|
||||
height: (ann_detail.height * height) / 100,
|
||||
Label: Array.isArray(ann_detail.rectanglelabels) ? (ann_detail.rectanglelabels[0] || 'unknown') : (ann_detail.rectanglelabels || 'unknown')
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 1) Insert images and get generated IDs
|
||||
const insertedImages = await Img.bulkCreate(imagesBulk, { returning: true });
|
||||
|
||||
// 2) Map image_path -> image_id
|
||||
const imageMap = {};
|
||||
for (const img of insertedImages) {
|
||||
imageMap[img.image_path] = img.image_id;
|
||||
}
|
||||
|
||||
// 3) Assign correct image_id to each annotation
|
||||
for (const ann of annsBulk) {
|
||||
ann.image_id = imageMap[ann.image_path];
|
||||
delete ann.image_path; // cleanup
|
||||
}
|
||||
|
||||
// 4) Insert annotations
|
||||
await Ann.bulkCreate(annsBulk);
|
||||
|
||||
console.log(`Inserted ${imagesBulk.length} images and ${annsBulk.length} annotations for project ${project.id}`);
|
||||
}
|
||||
|
||||
console.log('Seeding done');
|
||||
return { success: true, message: 'Data inserted successfully!' };
|
||||
} catch (error) {
|
||||
console.error('Error inserting data:', error);
|
||||
return { success: false, message: error.message };
|
||||
} finally {
|
||||
updateStatus.running = false;
|
||||
console.log('updateStatus.running set to false');
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { seedLabelStudio, updateStatus };
|
||||
0
backend/services/seed_label_studio.py
Normal file → Executable file
0
backend/services/seed_label_studio.py
Normal file → Executable file
0
backend/services/settings_service.py
Normal file → Executable file
0
backend/services/settings_service.py
Normal file → Executable file
206
backend/services/training_queue.py
Executable file
206
backend/services/training_queue.py
Executable file
@@ -0,0 +1,206 @@
|
||||
"""
|
||||
Training Queue Manager
|
||||
Manages a queue of training jobs and tracks their progress
|
||||
"""
|
||||
import threading
|
||||
import queue
|
||||
import subprocess
|
||||
import re
|
||||
import os
|
||||
from services.settings_service import get_setting
|
||||
from models.training import Training
|
||||
|
||||
class TrainingQueueManager:
|
||||
_instance = None
|
||||
_lock = threading.Lock()
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
with cls._lock:
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
cls._instance._initialized = False
|
||||
return cls._instance
|
||||
|
||||
def __init__(self):
|
||||
if self._initialized:
|
||||
return
|
||||
|
||||
self.queue = queue.Queue()
|
||||
self.current_training = None
|
||||
self.current_process = None
|
||||
self.worker_thread = None
|
||||
self.running = False
|
||||
self._initialized = True
|
||||
|
||||
# Start the worker thread
|
||||
self.start_worker()
|
||||
|
||||
def start_worker(self):
|
||||
"""Start the background worker thread"""
|
||||
if self.worker_thread is None or not self.worker_thread.is_alive():
|
||||
self.running = True
|
||||
self.worker_thread = threading.Thread(target=self._process_queue, daemon=True)
|
||||
self.worker_thread.start()
|
||||
|
||||
def add_to_queue(self, training_id, command, cwd):
|
||||
"""Add a training job to the queue"""
|
||||
job = {
|
||||
'training_id': training_id,
|
||||
'command': command,
|
||||
'cwd': cwd,
|
||||
'iteration': 0,
|
||||
'max_epoch': 300 # Will be updated from training record
|
||||
}
|
||||
|
||||
# Get max_epoch from training record
|
||||
try:
|
||||
training = Training.query.get(training_id)
|
||||
if training:
|
||||
job['max_epoch'] = training.max_epoch or 300
|
||||
job['name'] = training.exp_name or f'Training {training_id}'
|
||||
except:
|
||||
pass
|
||||
|
||||
self.queue.put(job)
|
||||
print(f'Added training {training_id} to queue. Queue size: {self.queue.qsize()}')
|
||||
|
||||
def _process_queue(self):
|
||||
"""Worker thread that processes the queue"""
|
||||
while self.running:
|
||||
try:
|
||||
# Wait for a job (blocking with timeout)
|
||||
job = self.queue.get(timeout=1)
|
||||
|
||||
print(f'Starting training {job["training_id"]} from queue')
|
||||
self.current_training = job
|
||||
|
||||
# Execute the training command
|
||||
self._run_training(job)
|
||||
|
||||
# Mark as done
|
||||
self.queue.task_done()
|
||||
self.current_training = None
|
||||
self.current_process = None
|
||||
|
||||
except queue.Empty:
|
||||
continue
|
||||
except Exception as e:
|
||||
print(f'Error processing training job: {e}')
|
||||
self.current_training = None
|
||||
self.current_process = None
|
||||
|
||||
def _run_training(self, job):
|
||||
"""Run a training command and monitor its output"""
|
||||
try:
|
||||
import platform
|
||||
is_windows = platform.system() == 'Windows'
|
||||
|
||||
# Start process
|
||||
self.current_process = subprocess.Popen(
|
||||
job['command'],
|
||||
shell=True,
|
||||
cwd=job['cwd'],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
universal_newlines=True,
|
||||
bufsize=1
|
||||
)
|
||||
|
||||
# Monitor output for progress
|
||||
for line in iter(self.current_process.stdout.readline, ''):
|
||||
if line:
|
||||
print(line.strip())
|
||||
|
||||
# Parse epoch and iteration from YOLOX output
|
||||
# Example: "epoch: 3/300, iter: 90/101"
|
||||
epoch_match = re.search(r'epoch:\s*(\d+)/(\d+)', line, re.IGNORECASE)
|
||||
iter_match = re.search(r'iter:\s*(\d+)/(\d+)', line, re.IGNORECASE)
|
||||
|
||||
if epoch_match:
|
||||
current_epoch = int(epoch_match.group(1))
|
||||
total_epochs = int(epoch_match.group(2))
|
||||
if self.current_training:
|
||||
self.current_training['current_epoch'] = current_epoch
|
||||
self.current_training['max_epoch'] = total_epochs
|
||||
# Debug log
|
||||
print(f'[PROGRESS] Parsed epoch: {current_epoch}/{total_epochs}')
|
||||
|
||||
if iter_match:
|
||||
current_iter = int(iter_match.group(1))
|
||||
total_iters = int(iter_match.group(2))
|
||||
if self.current_training:
|
||||
self.current_training['current_iter'] = current_iter
|
||||
self.current_training['total_iters'] = total_iters
|
||||
|
||||
# Calculate overall progress percentage
|
||||
if 'current_epoch' in self.current_training and 'max_epoch' in self.current_training:
|
||||
epoch_progress = (self.current_training['current_epoch'] - 1) / self.current_training['max_epoch']
|
||||
iter_progress = current_iter / total_iters / self.current_training['max_epoch']
|
||||
total_progress = (epoch_progress + iter_progress) * 100
|
||||
self.current_training['progress'] = round(total_progress, 2)
|
||||
# Debug log
|
||||
print(f'[PROGRESS] Epoch {self.current_training["current_epoch"]}/{self.current_training["max_epoch"]}, Iter {current_iter}/{total_iters}, Progress: {self.current_training["progress"]}%')
|
||||
|
||||
# Wait for completion
|
||||
self.current_process.wait()
|
||||
print(f'Training {job["training_id"]} completed with exit code {self.current_process.returncode}')
|
||||
|
||||
except Exception as e:
|
||||
print(f'Error running training: {e}')
|
||||
|
||||
def get_status(self):
|
||||
"""Get current status of training queue"""
|
||||
queue_items = []
|
||||
|
||||
# Get items from queue without removing them
|
||||
temp_items = []
|
||||
while not self.queue.empty():
|
||||
try:
|
||||
item = self.queue.get_nowait()
|
||||
temp_items.append(item)
|
||||
queue_items.append({
|
||||
'training_id': item['training_id'],
|
||||
'name': item.get('name', f'Training {item["training_id"]}'),
|
||||
'max_epoch': item.get('max_epoch', 300)
|
||||
})
|
||||
except queue.Empty:
|
||||
break
|
||||
|
||||
# Put items back
|
||||
for item in temp_items:
|
||||
self.queue.put(item)
|
||||
|
||||
result = {
|
||||
'current': None,
|
||||
'queue': queue_items
|
||||
}
|
||||
|
||||
if self.current_training:
|
||||
current_epoch = self.current_training.get('current_epoch', 0)
|
||||
max_epoch = self.current_training.get('max_epoch', 300)
|
||||
result['current'] = {
|
||||
'training_id': self.current_training['training_id'],
|
||||
'name': self.current_training.get('name', f'Training {self.current_training["training_id"]}'),
|
||||
'epoch': current_epoch, # For backward compatibility
|
||||
'current_epoch': current_epoch,
|
||||
'max_epoch': max_epoch,
|
||||
'current_iter': self.current_training.get('current_iter', 0),
|
||||
'total_iters': self.current_training.get('total_iters', 0),
|
||||
'progress': self.current_training.get('progress', 0.0),
|
||||
'iteration': current_epoch # For backward compatibility
|
||||
}
|
||||
|
||||
return result
|
||||
|
||||
def stop(self):
|
||||
"""Stop the worker thread"""
|
||||
self.running = False
|
||||
if self.current_process:
|
||||
try:
|
||||
self.current_process.terminate()
|
||||
except:
|
||||
pass
|
||||
|
||||
# Global instance
|
||||
training_queue = TrainingQueueManager()
|
||||
244
backend/services/validate_dataset.py
Executable file
244
backend/services/validate_dataset.py
Executable file
@@ -0,0 +1,244 @@
|
||||
"""
|
||||
Validate dataset for training - check for problematic images and annotations
|
||||
"""
|
||||
import os
|
||||
import json
|
||||
from PIL import Image
|
||||
import cv2
|
||||
|
||||
def validate_coco_json(json_path, data_dir):
|
||||
"""
|
||||
Validate a COCO JSON file and check all images
|
||||
|
||||
Args:
|
||||
json_path: Path to COCO JSON file
|
||||
data_dir: Directory where images are located
|
||||
|
||||
Returns:
|
||||
dict with validation results
|
||||
"""
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Validating: {json_path}")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
issues = {
|
||||
'missing_images': [],
|
||||
'corrupted_images': [],
|
||||
'zero_dimension_images': [],
|
||||
'invalid_annotations': [],
|
||||
'zero_area_boxes': []
|
||||
}
|
||||
|
||||
try:
|
||||
with open(json_path, 'r') as f:
|
||||
coco_data = json.load(f)
|
||||
except Exception as e:
|
||||
print(f"❌ Failed to load JSON: {e}")
|
||||
return issues
|
||||
|
||||
images = coco_data.get('images', [])
|
||||
annotations = coco_data.get('annotations', [])
|
||||
|
||||
print(f"📊 Dataset Stats:")
|
||||
print(f" Images: {len(images)}")
|
||||
print(f" Annotations: {len(annotations)}")
|
||||
print(f" Categories: {len(coco_data.get('categories', []))}")
|
||||
print()
|
||||
|
||||
# Validate images
|
||||
print("🔍 Validating images...")
|
||||
for idx, img_info in enumerate(images):
|
||||
img_id = img_info.get('id')
|
||||
file_name = img_info.get('file_name', '')
|
||||
width = img_info.get('width', 0)
|
||||
height = img_info.get('height', 0)
|
||||
|
||||
# Check if image file exists
|
||||
# Try to construct the full path
|
||||
if os.path.isabs(file_name):
|
||||
img_path = file_name
|
||||
else:
|
||||
img_path = os.path.join(data_dir, file_name)
|
||||
|
||||
if not os.path.exists(img_path):
|
||||
issues['missing_images'].append({
|
||||
'id': img_id,
|
||||
'file_name': file_name,
|
||||
'expected_path': img_path
|
||||
})
|
||||
continue
|
||||
|
||||
# Check if image can be loaded
|
||||
try:
|
||||
# Try with PIL
|
||||
with Image.open(img_path) as pil_img:
|
||||
pil_width, pil_height = pil_img.size
|
||||
|
||||
# Check if dimensions match JSON
|
||||
if pil_width != width or pil_height != height:
|
||||
print(f"⚠️ Image {img_id}: Dimension mismatch - JSON: {width}x{height}, Actual: {pil_width}x{pil_height}")
|
||||
|
||||
# Check for zero dimensions
|
||||
if pil_width == 0 or pil_height == 0:
|
||||
issues['zero_dimension_images'].append({
|
||||
'id': img_id,
|
||||
'file_name': file_name,
|
||||
'dimensions': f"{pil_width}x{pil_height}"
|
||||
})
|
||||
except Exception as e:
|
||||
issues['corrupted_images'].append({
|
||||
'id': img_id,
|
||||
'file_name': file_name,
|
||||
'error': str(e)
|
||||
})
|
||||
|
||||
# Progress indicator
|
||||
if (idx + 1) % 100 == 0:
|
||||
print(f" Checked {idx + 1}/{len(images)} images...")
|
||||
|
||||
print(f"✅ Image validation complete\n")
|
||||
|
||||
# Validate annotations
|
||||
print("🔍 Validating annotations...")
|
||||
for idx, ann in enumerate(annotations):
|
||||
ann_id = ann.get('id')
|
||||
img_id = ann.get('image_id')
|
||||
bbox = ann.get('bbox', [])
|
||||
|
||||
if len(bbox) != 4:
|
||||
issues['invalid_annotations'].append({
|
||||
'id': ann_id,
|
||||
'image_id': img_id,
|
||||
'reason': f'Invalid bbox length: {len(bbox)}'
|
||||
})
|
||||
continue
|
||||
|
||||
x, y, w, h = bbox
|
||||
|
||||
# Check for zero or negative dimensions
|
||||
if w <= 0 or h <= 0:
|
||||
issues['zero_area_boxes'].append({
|
||||
'id': ann_id,
|
||||
'image_id': img_id,
|
||||
'bbox': bbox,
|
||||
'reason': f'Zero or negative dimensions: w={w}, h={h}'
|
||||
})
|
||||
|
||||
# Check for extremely small boxes (potential issue with mixup)
|
||||
if w < 1 or h < 1:
|
||||
issues['zero_area_boxes'].append({
|
||||
'id': ann_id,
|
||||
'image_id': img_id,
|
||||
'bbox': bbox,
|
||||
'reason': f'Extremely small box: w={w}, h={h}'
|
||||
})
|
||||
|
||||
# Progress indicator
|
||||
if (idx + 1) % 1000 == 0:
|
||||
print(f" Checked {idx + 1}/{len(annotations)} annotations...")
|
||||
|
||||
print(f"✅ Annotation validation complete\n")
|
||||
|
||||
# Print summary
|
||||
print(f"\n{'='*60}")
|
||||
print("VALIDATION SUMMARY")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
total_issues = sum(len(v) for v in issues.values())
|
||||
|
||||
if total_issues == 0:
|
||||
print("✅ No issues found! Dataset is ready for training.")
|
||||
else:
|
||||
print(f"⚠️ Found {total_issues} total issues:\n")
|
||||
|
||||
if issues['missing_images']:
|
||||
print(f" ❌ Missing images: {len(issues['missing_images'])}")
|
||||
for item in issues['missing_images'][:5]: # Show first 5
|
||||
print(f" - {item['file_name']}")
|
||||
if len(issues['missing_images']) > 5:
|
||||
print(f" ... and {len(issues['missing_images']) - 5} more")
|
||||
|
||||
if issues['corrupted_images']:
|
||||
print(f" ❌ Corrupted images: {len(issues['corrupted_images'])}")
|
||||
for item in issues['corrupted_images'][:5]:
|
||||
print(f" - {item['file_name']}: {item['error']}")
|
||||
if len(issues['corrupted_images']) > 5:
|
||||
print(f" ... and {len(issues['corrupted_images']) - 5} more")
|
||||
|
||||
if issues['zero_dimension_images']:
|
||||
print(f" ❌ Zero dimension images: {len(issues['zero_dimension_images'])}")
|
||||
for item in issues['zero_dimension_images'][:5]:
|
||||
print(f" - {item['file_name']}: {item['dimensions']}")
|
||||
if len(issues['zero_dimension_images']) > 5:
|
||||
print(f" ... and {len(issues['zero_dimension_images']) - 5} more")
|
||||
|
||||
if issues['invalid_annotations']:
|
||||
print(f" ❌ Invalid annotations: {len(issues['invalid_annotations'])}")
|
||||
for item in issues['invalid_annotations'][:5]:
|
||||
print(f" - Ann ID {item['id']}: {item['reason']}")
|
||||
if len(issues['invalid_annotations']) > 5:
|
||||
print(f" ... and {len(issues['invalid_annotations']) - 5} more")
|
||||
|
||||
if issues['zero_area_boxes']:
|
||||
print(f" ⚠️ Zero/tiny area boxes: {len(issues['zero_area_boxes'])}")
|
||||
print(f" These may cause issues with mixup augmentation!")
|
||||
for item in issues['zero_area_boxes'][:5]:
|
||||
print(f" - Ann ID {item['id']}, bbox: {item['bbox']}")
|
||||
if len(issues['zero_area_boxes']) > 5:
|
||||
print(f" ... and {len(issues['zero_area_boxes']) - 5} more")
|
||||
|
||||
print()
|
||||
return issues
|
||||
|
||||
|
||||
def validate_training_dataset(training_id):
|
||||
"""
|
||||
Validate all COCO JSON files for a training
|
||||
|
||||
Args:
|
||||
training_id: The training ID to validate
|
||||
"""
|
||||
from models.training import Training
|
||||
from models.TrainingProject import TrainingProject
|
||||
from services.settings_service import get_setting
|
||||
|
||||
training = Training.query.get(training_id)
|
||||
if not training:
|
||||
print(f"❌ Training {training_id} not found")
|
||||
return
|
||||
|
||||
# Get paths
|
||||
from models.TrainingProjectDetails import TrainingProjectDetails
|
||||
details = TrainingProjectDetails.query.get(training.project_details_id)
|
||||
training_project = TrainingProject.query.get(details.project_id)
|
||||
project_name = training_project.title.replace(' ', '_') if training_project else f'project_{details.project_id}'
|
||||
|
||||
training_folder_name = f"{training.exp_name or training.training_name or 'training'}_{training_id}"
|
||||
training_folder_name = training_folder_name.replace(' ', '_')
|
||||
|
||||
output_base_path = get_setting('yolox_output_path', './backend')
|
||||
data_dir = get_setting('yolox_data_dir', '/home/kitraining/To_Annotate/')
|
||||
|
||||
annotations_dir = os.path.join(output_base_path, project_name, training_folder_name, 'annotations')
|
||||
|
||||
# Validate each split
|
||||
splits = ['train', 'valid', 'test']
|
||||
all_issues = {}
|
||||
|
||||
for split in splits:
|
||||
json_file = os.path.join(annotations_dir, f'coco_project_{training_id}_{split}.json')
|
||||
if os.path.exists(json_file):
|
||||
all_issues[split] = validate_coco_json(json_file, data_dir)
|
||||
else:
|
||||
print(f"⚠️ JSON file not found: {json_file}")
|
||||
|
||||
return all_issues
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
import sys
|
||||
if len(sys.argv) > 1:
|
||||
training_id = int(sys.argv[1])
|
||||
validate_training_dataset(training_id)
|
||||
else:
|
||||
print("Usage: python validate_dataset.py <training_id>")
|
||||
0
backend/start.py
Normal file → Executable file
0
backend/start.py
Normal file → Executable file
0
backend/test/7/exp.py
Normal file → Executable file
0
backend/test/7/exp.py
Normal file → Executable file
0
documentation/Projektdoku.pdf
Normal file → Executable file
0
documentation/Projektdoku.pdf
Normal file → Executable file
0
documentation/Projektdokumentation.md
Normal file → Executable file
0
documentation/Projektdokumentation.md
Normal file → Executable file
192
edit-training.html
Normal file → Executable file
192
edit-training.html
Normal file → Executable file
@@ -202,6 +202,14 @@
|
||||
<label id="project-title-label"
|
||||
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" onclick="window.location.href='/add-project.html'"
|
||||
class="button-red">Add Training Project</button>
|
||||
<button id="seed-db-btn" class="button">
|
||||
@@ -553,7 +561,7 @@ document.getElementById('parameters-form').addEventListener('submit', async func
|
||||
}
|
||||
// Add project id if available
|
||||
const urlParams = new URLSearchParams(window.location.search);
|
||||
const projectId = urlParams.get('id');
|
||||
const projectId = urlParams.get('project_id') || urlParams.get('id'); // Support both parameter names
|
||||
if (projectId) formData.append('project_id', projectId);
|
||||
// Send to backend
|
||||
try {
|
||||
@@ -640,4 +648,186 @@ document.getElementById('parameters-form').addEventListener('submit', async func
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
// Handle readonly mode and load training data
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
const urlParams = new URLSearchParams(window.location.search);
|
||||
const trainingId = urlParams.get('training_id');
|
||||
const isReadonly = urlParams.get('readonly') === 'true';
|
||||
|
||||
if (isReadonly) {
|
||||
// Update page title
|
||||
const titleLabel = document.querySelector('#project-title-label');
|
||||
if (titleLabel) {
|
||||
titleLabel.textContent = 'View Training Details (Read-Only)';
|
||||
}
|
||||
|
||||
// Disable all form inputs
|
||||
document.querySelectorAll('input, select, textarea, button[type="submit"]').forEach(el => {
|
||||
el.disabled = true;
|
||||
el.style.opacity = '0.6';
|
||||
el.style.cursor = 'not-allowed';
|
||||
});
|
||||
|
||||
// Hide submit buttons
|
||||
document.querySelectorAll('button[type="submit"]').forEach(btn => {
|
||||
btn.style.display = 'none';
|
||||
});
|
||||
|
||||
// Re-enable navigation buttons (back, etc.)
|
||||
document.querySelectorAll('button:not([type="submit"])').forEach(btn => {
|
||||
if (btn.textContent.includes('Back') || btn.onclick) {
|
||||
btn.disabled = false;
|
||||
btn.style.opacity = '1';
|
||||
btn.style.cursor = 'pointer';
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Load training data if training_id is present
|
||||
if (trainingId) {
|
||||
fetch(`/api/trainings/${trainingId}`)
|
||||
.then(res => res.json())
|
||||
.then(training => {
|
||||
// Populate form fields with training data
|
||||
Object.keys(training).forEach(key => {
|
||||
const input = document.querySelector(`[name="${key}"]`);
|
||||
if (input) {
|
||||
if (input.type === 'checkbox') {
|
||||
input.checked = training[key];
|
||||
} else {
|
||||
input.value = training[key] || '';
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// Handle special fields
|
||||
if (training.train) document.getElementById('train-slider').value = training.train;
|
||||
if (training.valid) document.getElementById('valid-slider').value = training.valid;
|
||||
if (training.test) document.getElementById('test-slider').value = training.test;
|
||||
syncSplitSliders('train');
|
||||
})
|
||||
.catch(err => {
|
||||
console.error('Failed to load training data:', err);
|
||||
});
|
||||
}
|
||||
});
|
||||
</script>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus();
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
</body></html>
|
||||
0
globals.css
Normal file → Executable file
0
globals.css
Normal file → Executable file
134
index.html
Normal file → Executable file
134
index.html
Normal file → Executable file
@@ -28,6 +28,14 @@
|
||||
<icon class="header-icon" onclick="window.location.href='/index.html'" , onmouseover="" style="cursor: pointer;"
|
||||
src="./media/logo.png" alt="Logo"></icon>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
|
||||
Training Project</button>
|
||||
<button id="seed-db-btn" class="button">
|
||||
@@ -159,7 +167,133 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<!-- Current Training -->
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Queued Trainings -->
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- No trainings message -->
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
// Training status polling
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus(); // Immediate update
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
// Update bell appearance
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
// Update modal content
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
// Poll every 5 seconds
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
// Stop polling when page unloads
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
0
js/add-class.js
Normal file → Executable file
0
js/add-class.js
Normal file → Executable file
0
js/add-image.js
Normal file → Executable file
0
js/add-image.js
Normal file → Executable file
0
js/dashboard-label-studio.js
Normal file → Executable file
0
js/dashboard-label-studio.js
Normal file → Executable file
0
js/dashboard.js
Normal file → Executable file
0
js/dashboard.js
Normal file → Executable file
0
js/overview-training.js
Normal file → Executable file
0
js/overview-training.js
Normal file → Executable file
0
js/settings.js
Normal file → Executable file
0
js/settings.js
Normal file → Executable file
0
js/setup-training-project.js
Normal file → Executable file
0
js/setup-training-project.js
Normal file → Executable file
0
js/start-training.js
Normal file → Executable file
0
js/start-training.js
Normal file → Executable file
0
js/storage.js
Normal file → Executable file
0
js/storage.js
Normal file → Executable file
0
media/logo.png
Normal file → Executable file
0
media/logo.png
Normal file → Executable file
|
Before Width: | Height: | Size: 11 KiB After Width: | Height: | Size: 11 KiB |
154
overview-training.html
Normal file → Executable file
154
overview-training.html
Normal file → Executable file
@@ -30,6 +30,14 @@
|
||||
<label id="project-title-label"
|
||||
style="display: block; text-align: left; font-weight: bold; font-size: x-large;">Project</label>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
|
||||
Training Project</button>
|
||||
<button id="seed-db-btn" class="button">
|
||||
@@ -44,10 +52,7 @@
|
||||
</div>
|
||||
</button>
|
||||
|
||||
</button>
|
||||
<button id="setup-details" class="button">
|
||||
Show Details
|
||||
</button>
|
||||
|
||||
|
||||
<script>
|
||||
document.getElementById('seed-db-btn').addEventListener('click', function () {
|
||||
@@ -156,14 +161,15 @@
|
||||
};
|
||||
btnDiv.appendChild(startBtn);
|
||||
|
||||
// View Log button
|
||||
const logBtn = document.createElement('button');
|
||||
logBtn.textContent = 'View Training Log';
|
||||
logBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
|
||||
logBtn.onclick = function() {
|
||||
showLogModal(training.id);
|
||||
// View Training Details button
|
||||
const detailsBtn = document.createElement('button');
|
||||
detailsBtn.textContent = 'View Training Details';
|
||||
detailsBtn.style = 'background:#666;color:white;border:none;border-radius:6px;padding:6px 12px;cursor:pointer;';
|
||||
detailsBtn.onclick = function() {
|
||||
// Navigate to edit-training page in read-only mode
|
||||
window.location.href = `/edit-training.html?training_id=${training.id}&readonly=true`;
|
||||
};
|
||||
btnDiv.appendChild(logBtn);
|
||||
btnDiv.appendChild(detailsBtn);
|
||||
|
||||
// Remove button
|
||||
const removeBtn = document.createElement('button');
|
||||
@@ -319,7 +325,133 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<!-- Current Training -->
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Queued Trainings -->
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- No trainings message -->
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
// Training status polling
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus(); // Immediate update
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
// Update bell appearance
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
// Update modal content
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
// Poll every 5 seconds
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
// Stop polling when page unloads
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
124
project-details.html
Normal file → Executable file
124
project-details.html
Normal file → Executable file
@@ -32,6 +32,14 @@
|
||||
style="cursor: pointer;" src="./media/logo.png" alt="Logo"></icon>
|
||||
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" onclick="window.location.href='/add-project.html'"
|
||||
class="button-red">Add Training Project</button>
|
||||
<button id="seed-db-btn" class="button">
|
||||
@@ -177,7 +185,123 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus();
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
0
settings.html
Normal file → Executable file
0
settings.html
Normal file → Executable file
130
setup-training-project.html
Normal file → Executable file
130
setup-training-project.html
Normal file → Executable file
@@ -29,6 +29,14 @@
|
||||
src="./media/logo.png" alt="Logo"></icon>
|
||||
<label id="project-title-label" style="display: block; text-align: left; font-weight: bold; font-size: x-large;">title</label>
|
||||
<div class="button-row">
|
||||
<!-- Training Notification Bell -->
|
||||
<button id="training-bell" onclick="toggleTrainingModal()" class="button" title="Training Status"
|
||||
style="padding: 8px 16px; margin-right: 10px; position: relative; background: #999;">
|
||||
🔔
|
||||
<span id="bell-badge" style="display: none; position: absolute; top: -5px; right: -5px; background: #ff4d4f;
|
||||
color: white; border-radius: 50%; width: 20px; height: 20px; font-size: 12px; line-height: 20px;
|
||||
text-align: center; font-weight: bold;">0</span>
|
||||
</button>
|
||||
<button id="Add Training Project" onclick="window.location.href='/add-project.html'" class="button-red">Add
|
||||
Training Project</button>
|
||||
<button id="seed-db-btn" class="button">
|
||||
@@ -170,7 +178,129 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Training Status Modal -->
|
||||
<div id="training-status-modal" class="modal" style="display: none;">
|
||||
<div class="modal-content" style="max-width: 700px; max-height: 90vh; overflow-y: auto;">
|
||||
<div class="modal-header">
|
||||
<h2>Training Status</h2>
|
||||
<button class="close-btn" onclick="toggleTrainingModal()">×</button>
|
||||
</div>
|
||||
|
||||
<div class="modal-body">
|
||||
<!-- Current Training -->
|
||||
<div class="settings-section" id="current-training-section" style="display: none;">
|
||||
<h3 style="color: #009eac;">Current Training</h3>
|
||||
<div id="current-training-info" style="background: #eaf7fa; padding: 16px; border-radius: 8px; margin-bottom: 16px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Queued Trainings -->
|
||||
<div class="settings-section" id="queue-section" style="display: none;">
|
||||
<h3 style="color: #666;">Queue (<span id="queue-count">0</span>)</h3>
|
||||
<div id="queue-list" style="display: flex; flex-direction: column; gap: 12px;">
|
||||
<!-- Populated by JS -->
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- No trainings message -->
|
||||
<div id="no-trainings-msg" style="text-align: center; padding: 32px; color: #666;">
|
||||
No trainings running or queued.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="js/settings.js"></script>
|
||||
<script>
|
||||
// Training status polling
|
||||
let trainingStatusPoller = null;
|
||||
|
||||
function toggleTrainingModal() {
|
||||
const modal = document.getElementById('training-status-modal');
|
||||
if (modal.style.display === 'none') {
|
||||
modal.style.display = 'flex';
|
||||
updateTrainingStatus();
|
||||
} else {
|
||||
modal.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
function updateTrainingStatus() {
|
||||
fetch('/api/training-status')
|
||||
.then(res => res.json())
|
||||
.then(data => {
|
||||
const bell = document.getElementById('training-bell');
|
||||
const badge = document.getElementById('bell-badge');
|
||||
const currentSection = document.getElementById('current-training-section');
|
||||
const queueSection = document.getElementById('queue-section');
|
||||
const noTrainingsMsg = document.getElementById('no-trainings-msg');
|
||||
|
||||
const totalCount = (data.current ? 1 : 0) + data.queue.length;
|
||||
|
||||
if (totalCount > 0) {
|
||||
bell.style.background = '#009eac';
|
||||
badge.style.display = 'block';
|
||||
badge.textContent = totalCount;
|
||||
} else {
|
||||
bell.style.background = '#999';
|
||||
badge.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.current) {
|
||||
currentSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
|
||||
const percentage = Math.round((data.current.iteration / data.current.max_epoch) * 100);
|
||||
document.getElementById('current-training-info').innerHTML = `
|
||||
<div style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 8px;">
|
||||
<strong>${data.current.name || 'Training'}</strong>
|
||||
<span style="font-weight: bold; color: #009eac;">${percentage}%</span>
|
||||
</div>
|
||||
<div style="background: #ddd; border-radius: 4px; height: 24px; overflow: hidden; margin-bottom: 8px;">
|
||||
<div style="background: #009eac; height: 100%; width: ${percentage}%; transition: width 0.3s;"></div>
|
||||
</div>
|
||||
<div style="font-size: 14px; color: #666;">
|
||||
Epoch ${data.current.iteration} / ${data.current.max_epoch}
|
||||
</div>
|
||||
`;
|
||||
} else {
|
||||
currentSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (data.queue.length > 0) {
|
||||
queueSection.style.display = 'block';
|
||||
noTrainingsMsg.style.display = 'none';
|
||||
document.getElementById('queue-count').textContent = data.queue.length;
|
||||
|
||||
document.getElementById('queue-list').innerHTML = data.queue.map((t, idx) => `
|
||||
<div style="background: #f5f5f5; padding: 12px; border-radius: 8px; border-left: 4px solid #009eac;">
|
||||
<strong>#${idx + 1}: ${t.name || 'Training'}</strong>
|
||||
<div style="font-size: 13px; color: #666; margin-top: 4px;">
|
||||
${t.max_epoch} epochs • Waiting...
|
||||
</div>
|
||||
</div>
|
||||
`).join('');
|
||||
} else {
|
||||
queueSection.style.display = 'none';
|
||||
}
|
||||
|
||||
if (totalCount === 0) {
|
||||
noTrainingsMsg.style.display = 'block';
|
||||
}
|
||||
})
|
||||
.catch(err => console.error('Failed to fetch training status:', err));
|
||||
}
|
||||
|
||||
window.addEventListener('DOMContentLoaded', function() {
|
||||
updateTrainingStatus();
|
||||
trainingStatusPoller = setInterval(updateTrainingStatus, 5000);
|
||||
});
|
||||
|
||||
window.addEventListener('beforeunload', function() {
|
||||
if (trainingStatusPoller) clearInterval(trainingStatusPoller);
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
|
||||
</html>
|
||||
0
start-training.html
Normal file → Executable file
0
start-training.html
Normal file → Executable file
0
styleguide.css
Normal file → Executable file
0
styleguide.css
Normal file → Executable file
247
venv/bin/Activate.ps1
Executable file
247
venv/bin/Activate.ps1
Executable file
@@ -0,0 +1,247 @@
|
||||
<#
|
||||
.Synopsis
|
||||
Activate a Python virtual environment for the current PowerShell session.
|
||||
|
||||
.Description
|
||||
Pushes the python executable for a virtual environment to the front of the
|
||||
$Env:PATH environment variable and sets the prompt to signify that you are
|
||||
in a Python virtual environment. Makes use of the command line switches as
|
||||
well as the `pyvenv.cfg` file values present in the virtual environment.
|
||||
|
||||
.Parameter VenvDir
|
||||
Path to the directory that contains the virtual environment to activate. The
|
||||
default value for this is the parent of the directory that the Activate.ps1
|
||||
script is located within.
|
||||
|
||||
.Parameter Prompt
|
||||
The prompt prefix to display when this virtual environment is activated. By
|
||||
default, this prompt is the name of the virtual environment folder (VenvDir)
|
||||
surrounded by parentheses and followed by a single space (ie. '(.venv) ').
|
||||
|
||||
.Example
|
||||
Activate.ps1
|
||||
Activates the Python virtual environment that contains the Activate.ps1 script.
|
||||
|
||||
.Example
|
||||
Activate.ps1 -Verbose
|
||||
Activates the Python virtual environment that contains the Activate.ps1 script,
|
||||
and shows extra information about the activation as it executes.
|
||||
|
||||
.Example
|
||||
Activate.ps1 -VenvDir C:\Users\MyUser\Common\.venv
|
||||
Activates the Python virtual environment located in the specified location.
|
||||
|
||||
.Example
|
||||
Activate.ps1 -Prompt "MyPython"
|
||||
Activates the Python virtual environment that contains the Activate.ps1 script,
|
||||
and prefixes the current prompt with the specified string (surrounded in
|
||||
parentheses) while the virtual environment is active.
|
||||
|
||||
.Notes
|
||||
On Windows, it may be required to enable this Activate.ps1 script by setting the
|
||||
execution policy for the user. You can do this by issuing the following PowerShell
|
||||
command:
|
||||
|
||||
PS C:\> Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
|
||||
|
||||
For more information on Execution Policies:
|
||||
https://go.microsoft.com/fwlink/?LinkID=135170
|
||||
|
||||
#>
|
||||
Param(
|
||||
[Parameter(Mandatory = $false)]
|
||||
[String]
|
||||
$VenvDir,
|
||||
[Parameter(Mandatory = $false)]
|
||||
[String]
|
||||
$Prompt
|
||||
)
|
||||
|
||||
<# Function declarations --------------------------------------------------- #>
|
||||
|
||||
<#
|
||||
.Synopsis
|
||||
Remove all shell session elements added by the Activate script, including the
|
||||
addition of the virtual environment's Python executable from the beginning of
|
||||
the PATH variable.
|
||||
|
||||
.Parameter NonDestructive
|
||||
If present, do not remove this function from the global namespace for the
|
||||
session.
|
||||
|
||||
#>
|
||||
function global:deactivate ([switch]$NonDestructive) {
|
||||
# Revert to original values
|
||||
|
||||
# The prior prompt:
|
||||
if (Test-Path -Path Function:_OLD_VIRTUAL_PROMPT) {
|
||||
Copy-Item -Path Function:_OLD_VIRTUAL_PROMPT -Destination Function:prompt
|
||||
Remove-Item -Path Function:_OLD_VIRTUAL_PROMPT
|
||||
}
|
||||
|
||||
# The prior PYTHONHOME:
|
||||
if (Test-Path -Path Env:_OLD_VIRTUAL_PYTHONHOME) {
|
||||
Copy-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME -Destination Env:PYTHONHOME
|
||||
Remove-Item -Path Env:_OLD_VIRTUAL_PYTHONHOME
|
||||
}
|
||||
|
||||
# The prior PATH:
|
||||
if (Test-Path -Path Env:_OLD_VIRTUAL_PATH) {
|
||||
Copy-Item -Path Env:_OLD_VIRTUAL_PATH -Destination Env:PATH
|
||||
Remove-Item -Path Env:_OLD_VIRTUAL_PATH
|
||||
}
|
||||
|
||||
# Just remove the VIRTUAL_ENV altogether:
|
||||
if (Test-Path -Path Env:VIRTUAL_ENV) {
|
||||
Remove-Item -Path env:VIRTUAL_ENV
|
||||
}
|
||||
|
||||
# Just remove VIRTUAL_ENV_PROMPT altogether.
|
||||
if (Test-Path -Path Env:VIRTUAL_ENV_PROMPT) {
|
||||
Remove-Item -Path env:VIRTUAL_ENV_PROMPT
|
||||
}
|
||||
|
||||
# Just remove the _PYTHON_VENV_PROMPT_PREFIX altogether:
|
||||
if (Get-Variable -Name "_PYTHON_VENV_PROMPT_PREFIX" -ErrorAction SilentlyContinue) {
|
||||
Remove-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Scope Global -Force
|
||||
}
|
||||
|
||||
# Leave deactivate function in the global namespace if requested:
|
||||
if (-not $NonDestructive) {
|
||||
Remove-Item -Path function:deactivate
|
||||
}
|
||||
}
|
||||
|
||||
<#
|
||||
.Description
|
||||
Get-PyVenvConfig parses the values from the pyvenv.cfg file located in the
|
||||
given folder, and returns them in a map.
|
||||
|
||||
For each line in the pyvenv.cfg file, if that line can be parsed into exactly
|
||||
two strings separated by `=` (with any amount of whitespace surrounding the =)
|
||||
then it is considered a `key = value` line. The left hand string is the key,
|
||||
the right hand is the value.
|
||||
|
||||
If the value starts with a `'` or a `"` then the first and last character is
|
||||
stripped from the value before being captured.
|
||||
|
||||
.Parameter ConfigDir
|
||||
Path to the directory that contains the `pyvenv.cfg` file.
|
||||
#>
|
||||
function Get-PyVenvConfig(
|
||||
[String]
|
||||
$ConfigDir
|
||||
) {
|
||||
Write-Verbose "Given ConfigDir=$ConfigDir, obtain values in pyvenv.cfg"
|
||||
|
||||
# Ensure the file exists, and issue a warning if it doesn't (but still allow the function to continue).
|
||||
$pyvenvConfigPath = Join-Path -Resolve -Path $ConfigDir -ChildPath 'pyvenv.cfg' -ErrorAction Continue
|
||||
|
||||
# An empty map will be returned if no config file is found.
|
||||
$pyvenvConfig = @{ }
|
||||
|
||||
if ($pyvenvConfigPath) {
|
||||
|
||||
Write-Verbose "File exists, parse `key = value` lines"
|
||||
$pyvenvConfigContent = Get-Content -Path $pyvenvConfigPath
|
||||
|
||||
$pyvenvConfigContent | ForEach-Object {
|
||||
$keyval = $PSItem -split "\s*=\s*", 2
|
||||
if ($keyval[0] -and $keyval[1]) {
|
||||
$val = $keyval[1]
|
||||
|
||||
# Remove extraneous quotations around a string value.
|
||||
if ("'""".Contains($val.Substring(0, 1))) {
|
||||
$val = $val.Substring(1, $val.Length - 2)
|
||||
}
|
||||
|
||||
$pyvenvConfig[$keyval[0]] = $val
|
||||
Write-Verbose "Adding Key: '$($keyval[0])'='$val'"
|
||||
}
|
||||
}
|
||||
}
|
||||
return $pyvenvConfig
|
||||
}
|
||||
|
||||
|
||||
<# Begin Activate script --------------------------------------------------- #>
|
||||
|
||||
# Determine the containing directory of this script
|
||||
$VenvExecPath = Split-Path -Parent $MyInvocation.MyCommand.Definition
|
||||
$VenvExecDir = Get-Item -Path $VenvExecPath
|
||||
|
||||
Write-Verbose "Activation script is located in path: '$VenvExecPath'"
|
||||
Write-Verbose "VenvExecDir Fullname: '$($VenvExecDir.FullName)"
|
||||
Write-Verbose "VenvExecDir Name: '$($VenvExecDir.Name)"
|
||||
|
||||
# Set values required in priority: CmdLine, ConfigFile, Default
|
||||
# First, get the location of the virtual environment, it might not be
|
||||
# VenvExecDir if specified on the command line.
|
||||
if ($VenvDir) {
|
||||
Write-Verbose "VenvDir given as parameter, using '$VenvDir' to determine values"
|
||||
}
|
||||
else {
|
||||
Write-Verbose "VenvDir not given as a parameter, using parent directory name as VenvDir."
|
||||
$VenvDir = $VenvExecDir.Parent.FullName.TrimEnd("\\/")
|
||||
Write-Verbose "VenvDir=$VenvDir"
|
||||
}
|
||||
|
||||
# Next, read the `pyvenv.cfg` file to determine any required value such
|
||||
# as `prompt`.
|
||||
$pyvenvCfg = Get-PyVenvConfig -ConfigDir $VenvDir
|
||||
|
||||
# Next, set the prompt from the command line, or the config file, or
|
||||
# just use the name of the virtual environment folder.
|
||||
if ($Prompt) {
|
||||
Write-Verbose "Prompt specified as argument, using '$Prompt'"
|
||||
}
|
||||
else {
|
||||
Write-Verbose "Prompt not specified as argument to script, checking pyvenv.cfg value"
|
||||
if ($pyvenvCfg -and $pyvenvCfg['prompt']) {
|
||||
Write-Verbose " Setting based on value in pyvenv.cfg='$($pyvenvCfg['prompt'])'"
|
||||
$Prompt = $pyvenvCfg['prompt'];
|
||||
}
|
||||
else {
|
||||
Write-Verbose " Setting prompt based on parent's directory's name. (Is the directory name passed to venv module when creating the virtual environment)"
|
||||
Write-Verbose " Got leaf-name of $VenvDir='$(Split-Path -Path $venvDir -Leaf)'"
|
||||
$Prompt = Split-Path -Path $venvDir -Leaf
|
||||
}
|
||||
}
|
||||
|
||||
Write-Verbose "Prompt = '$Prompt'"
|
||||
Write-Verbose "VenvDir='$VenvDir'"
|
||||
|
||||
# Deactivate any currently active virtual environment, but leave the
|
||||
# deactivate function in place.
|
||||
deactivate -nondestructive
|
||||
|
||||
# Now set the environment variable VIRTUAL_ENV, used by many tools to determine
|
||||
# that there is an activated venv.
|
||||
$env:VIRTUAL_ENV = $VenvDir
|
||||
|
||||
if (-not $Env:VIRTUAL_ENV_DISABLE_PROMPT) {
|
||||
|
||||
Write-Verbose "Setting prompt to '$Prompt'"
|
||||
|
||||
# Set the prompt to include the env name
|
||||
# Make sure _OLD_VIRTUAL_PROMPT is global
|
||||
function global:_OLD_VIRTUAL_PROMPT { "" }
|
||||
Copy-Item -Path function:prompt -Destination function:_OLD_VIRTUAL_PROMPT
|
||||
New-Variable -Name _PYTHON_VENV_PROMPT_PREFIX -Description "Python virtual environment prompt prefix" -Scope Global -Option ReadOnly -Visibility Public -Value $Prompt
|
||||
|
||||
function global:prompt {
|
||||
Write-Host -NoNewline -ForegroundColor Green "($_PYTHON_VENV_PROMPT_PREFIX) "
|
||||
_OLD_VIRTUAL_PROMPT
|
||||
}
|
||||
$env:VIRTUAL_ENV_PROMPT = $Prompt
|
||||
}
|
||||
|
||||
# Clear PYTHONHOME
|
||||
if (Test-Path -Path Env:PYTHONHOME) {
|
||||
Copy-Item -Path Env:PYTHONHOME -Destination Env:_OLD_VIRTUAL_PYTHONHOME
|
||||
Remove-Item -Path Env:PYTHONHOME
|
||||
}
|
||||
|
||||
# Add the venv to the PATH
|
||||
Copy-Item -Path Env:PATH -Destination Env:_OLD_VIRTUAL_PATH
|
||||
$Env:PATH = "$VenvExecDir$([System.IO.Path]::PathSeparator)$Env:PATH"
|
||||
69
venv/bin/activate
Executable file
69
venv/bin/activate
Executable file
@@ -0,0 +1,69 @@
|
||||
# This file must be used with "source bin/activate" *from bash*
|
||||
# you cannot run it directly
|
||||
|
||||
deactivate () {
|
||||
# reset old environment variables
|
||||
if [ -n "${_OLD_VIRTUAL_PATH:-}" ] ; then
|
||||
PATH="${_OLD_VIRTUAL_PATH:-}"
|
||||
export PATH
|
||||
unset _OLD_VIRTUAL_PATH
|
||||
fi
|
||||
if [ -n "${_OLD_VIRTUAL_PYTHONHOME:-}" ] ; then
|
||||
PYTHONHOME="${_OLD_VIRTUAL_PYTHONHOME:-}"
|
||||
export PYTHONHOME
|
||||
unset _OLD_VIRTUAL_PYTHONHOME
|
||||
fi
|
||||
|
||||
# This should detect bash and zsh, which have a hash command that must
|
||||
# be called to get it to forget past commands. Without forgetting
|
||||
# past commands the $PATH changes we made may not be respected
|
||||
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
|
||||
hash -r 2> /dev/null
|
||||
fi
|
||||
|
||||
if [ -n "${_OLD_VIRTUAL_PS1:-}" ] ; then
|
||||
PS1="${_OLD_VIRTUAL_PS1:-}"
|
||||
export PS1
|
||||
unset _OLD_VIRTUAL_PS1
|
||||
fi
|
||||
|
||||
unset VIRTUAL_ENV
|
||||
unset VIRTUAL_ENV_PROMPT
|
||||
if [ ! "${1:-}" = "nondestructive" ] ; then
|
||||
# Self destruct!
|
||||
unset -f deactivate
|
||||
fi
|
||||
}
|
||||
|
||||
# unset irrelevant variables
|
||||
deactivate nondestructive
|
||||
|
||||
VIRTUAL_ENV=/home/kitraining/coco-tool/Abschluss-Projekt/venv
|
||||
export VIRTUAL_ENV
|
||||
|
||||
_OLD_VIRTUAL_PATH="$PATH"
|
||||
PATH="$VIRTUAL_ENV/"bin":$PATH"
|
||||
export PATH
|
||||
|
||||
# unset PYTHONHOME if set
|
||||
# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
|
||||
# could use `if (set -u; : $PYTHONHOME) ;` in bash
|
||||
if [ -n "${PYTHONHOME:-}" ] ; then
|
||||
_OLD_VIRTUAL_PYTHONHOME="${PYTHONHOME:-}"
|
||||
unset PYTHONHOME
|
||||
fi
|
||||
|
||||
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT:-}" ] ; then
|
||||
_OLD_VIRTUAL_PS1="${PS1:-}"
|
||||
PS1='(venv) '"${PS1:-}"
|
||||
export PS1
|
||||
VIRTUAL_ENV_PROMPT='(venv) '
|
||||
export VIRTUAL_ENV_PROMPT
|
||||
fi
|
||||
|
||||
# This should detect bash and zsh, which have a hash command that must
|
||||
# be called to get it to forget past commands. Without forgetting
|
||||
# past commands the $PATH changes we made may not be respected
|
||||
if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
|
||||
hash -r 2> /dev/null
|
||||
fi
|
||||
26
venv/bin/activate.csh
Executable file
26
venv/bin/activate.csh
Executable file
@@ -0,0 +1,26 @@
|
||||
# This file must be used with "source bin/activate.csh" *from csh*.
|
||||
# You cannot run it directly.
|
||||
# Created by Davide Di Blasi <davidedb@gmail.com>.
|
||||
# Ported to Python 3.3 venv by Andrew Svetlov <andrew.svetlov@gmail.com>
|
||||
|
||||
alias deactivate 'test $?_OLD_VIRTUAL_PATH != 0 && setenv PATH "$_OLD_VIRTUAL_PATH" && unset _OLD_VIRTUAL_PATH; rehash; test $?_OLD_VIRTUAL_PROMPT != 0 && set prompt="$_OLD_VIRTUAL_PROMPT" && unset _OLD_VIRTUAL_PROMPT; unsetenv VIRTUAL_ENV; unsetenv VIRTUAL_ENV_PROMPT; test "\!:*" != "nondestructive" && unalias deactivate'
|
||||
|
||||
# Unset irrelevant variables.
|
||||
deactivate nondestructive
|
||||
|
||||
setenv VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
|
||||
|
||||
set _OLD_VIRTUAL_PATH="$PATH"
|
||||
setenv PATH "$VIRTUAL_ENV/"bin":$PATH"
|
||||
|
||||
|
||||
set _OLD_VIRTUAL_PROMPT="$prompt"
|
||||
|
||||
if (! "$?VIRTUAL_ENV_DISABLE_PROMPT") then
|
||||
set prompt = '(venv) '"$prompt"
|
||||
setenv VIRTUAL_ENV_PROMPT '(venv) '
|
||||
endif
|
||||
|
||||
alias pydoc python -m pydoc
|
||||
|
||||
rehash
|
||||
69
venv/bin/activate.fish
Executable file
69
venv/bin/activate.fish
Executable file
@@ -0,0 +1,69 @@
|
||||
# This file must be used with "source <venv>/bin/activate.fish" *from fish*
|
||||
# (https://fishshell.com/); you cannot run it directly.
|
||||
|
||||
function deactivate -d "Exit virtual environment and return to normal shell environment"
|
||||
# reset old environment variables
|
||||
if test -n "$_OLD_VIRTUAL_PATH"
|
||||
set -gx PATH $_OLD_VIRTUAL_PATH
|
||||
set -e _OLD_VIRTUAL_PATH
|
||||
end
|
||||
if test -n "$_OLD_VIRTUAL_PYTHONHOME"
|
||||
set -gx PYTHONHOME $_OLD_VIRTUAL_PYTHONHOME
|
||||
set -e _OLD_VIRTUAL_PYTHONHOME
|
||||
end
|
||||
|
||||
if test -n "$_OLD_FISH_PROMPT_OVERRIDE"
|
||||
set -e _OLD_FISH_PROMPT_OVERRIDE
|
||||
# prevents error when using nested fish instances (Issue #93858)
|
||||
if functions -q _old_fish_prompt
|
||||
functions -e fish_prompt
|
||||
functions -c _old_fish_prompt fish_prompt
|
||||
functions -e _old_fish_prompt
|
||||
end
|
||||
end
|
||||
|
||||
set -e VIRTUAL_ENV
|
||||
set -e VIRTUAL_ENV_PROMPT
|
||||
if test "$argv[1]" != "nondestructive"
|
||||
# Self-destruct!
|
||||
functions -e deactivate
|
||||
end
|
||||
end
|
||||
|
||||
# Unset irrelevant variables.
|
||||
deactivate nondestructive
|
||||
|
||||
set -gx VIRTUAL_ENV /home/kitraining/coco-tool/Abschluss-Projekt/venv
|
||||
|
||||
set -gx _OLD_VIRTUAL_PATH $PATH
|
||||
set -gx PATH "$VIRTUAL_ENV/"bin $PATH
|
||||
|
||||
# Unset PYTHONHOME if set.
|
||||
if set -q PYTHONHOME
|
||||
set -gx _OLD_VIRTUAL_PYTHONHOME $PYTHONHOME
|
||||
set -e PYTHONHOME
|
||||
end
|
||||
|
||||
if test -z "$VIRTUAL_ENV_DISABLE_PROMPT"
|
||||
# fish uses a function instead of an env var to generate the prompt.
|
||||
|
||||
# Save the current fish_prompt function as the function _old_fish_prompt.
|
||||
functions -c fish_prompt _old_fish_prompt
|
||||
|
||||
# With the original prompt function renamed, we can override with our own.
|
||||
function fish_prompt
|
||||
# Save the return status of the last command.
|
||||
set -l old_status $status
|
||||
|
||||
# Output the venv prompt; color taken from the blue of the Python logo.
|
||||
printf "%s%s%s" (set_color 4B8BBE) '(venv) ' (set_color normal)
|
||||
|
||||
# Restore the return status of the previous command.
|
||||
echo "exit $old_status" | .
|
||||
# Output the original/"old" prompt.
|
||||
_old_fish_prompt
|
||||
end
|
||||
|
||||
set -gx _OLD_FISH_PROMPT_OVERRIDE "$VIRTUAL_ENV"
|
||||
set -gx VIRTUAL_ENV_PROMPT '(venv) '
|
||||
end
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user