11 changed files with 1506 additions and 277 deletions
@ -1,51 +1,85 @@ |
|||||||
# PROJECT_STATE.md |
# PROJECT_STATE.md |
||||||
|
|
||||||
Project: OTB Cloud |
Project: OTB Cloud |
||||||
Version: v1.1.0-alpha3 |
Version: v1.1.0-alpha4 |
||||||
Updated: 2026-04-19 |
Updated: 2026-04-20 |
||||||
Location: /opt/otb_cloud |
Location: /opt/otb_cloud |
||||||
|
|
||||||
## Current State |
## Current State |
||||||
OTB Cloud now has a functioning workshop-driven video processing pipeline. |
OTB Cloud now has a working multi-profile, multi-GPU video processing pipeline integrated into the tenant storage platform. |
||||||
|
|
||||||
### Confirmed Working |
### Confirmed Working |
||||||
- Portal and branded UI shell |
- Portal-branded OTB Cloud dashboard |
||||||
- Device browser |
- Device creation and browsing |
||||||
- File selection flow into Video Workshop |
- File-ID based workshop staging |
||||||
|
- Device-specific Video Workspace access from dashboard |
||||||
- Video Workshop page |
- Video Workshop page |
||||||
- Enqueue API |
- Multi-profile processing selection: |
||||||
- Jobs API |
- default |
||||||
- MariaDB-backed video_jobs integration |
- compress |
||||||
- Tenant/device path resolution for queued jobs |
- hq |
||||||
- Worker service startup and queue pickup |
- Manual rotation override option: |
||||||
- Worker-side absolute path resolution from tenant storage_root |
- auto/default behavior when unchecked |
||||||
- Intel iGPU processing path |
- selectable 90 / 180 / 270 override when enabled |
||||||
- Successful completed output for device 27 (ripper) |
- Job queue API and job listing API |
||||||
|
- File-ID based source resolution |
||||||
### Latest Proven Result |
- Output routing to: |
||||||
A queued workshop job for: |
- devices/<device>/video/ |
||||||
- source file: 05142013003.mp4 |
- Profile-specific output filenames |
||||||
- device: 27 (ripper) |
- Completed job actions: |
||||||
|
- View |
||||||
completed successfully with: |
- Send to LTS |
||||||
- assigned_processor: intel |
- Download Output |
||||||
- status: complete |
- Delete |
||||||
- progress_percent: 100 |
- Failed job delete action |
||||||
- output_relative_path: |
- LTS routing by file type: |
||||||
devices/ripper/originals/20260413T210325474049Z__05142013003_processed.mp4 |
- lts/video |
||||||
|
- lts/archived |
||||||
|
- lts/pictures |
||||||
|
- Health page |
||||||
|
- Lifetime processing metrics retained after visible job deletion |
||||||
|
- Intel + AMD GPU processing both in service |
||||||
|
- GPU time accounting active in Health page |
||||||
|
- Global video jobs route exists in codebase |
||||||
|
- Processed video section exists in device browser flow |
||||||
|
|
||||||
|
### Processing / GPU Behavior |
||||||
|
Current live behavior: |
||||||
|
- both GPUs take jobs |
||||||
|
- AMD prioritizes heavier / HQ work first |
||||||
|
- Intel handles lighter work |
||||||
|
- workers continue taking suitable jobs from the queue batch as available |
||||||
|
|
||||||
|
### Latest Proven Health State |
||||||
|
Health page currently shows stable cumulative values including: |
||||||
|
- uploaded file counts and space |
||||||
|
- LTS counts and space |
||||||
|
- archive counts and space |
||||||
|
- total jobs |
||||||
|
- completed jobs |
||||||
|
- failed jobs |
||||||
|
- cumulative GPU time that does not zero out when workshop cards are deleted |
||||||
|
|
||||||
|
### Current Storage Layout |
||||||
|
- originals remain in device originals tree |
||||||
|
- processed outputs go to: |
||||||
|
- devices/<device>/video/ |
||||||
|
- LTS destinations include: |
||||||
|
- lts/video/ |
||||||
|
- lts/archived/ |
||||||
|
- lts/pictures/ |
||||||
|
|
||||||
## Known Remaining Improvements |
## Known Remaining Improvements |
||||||
- Jobs panel is still raw JSON instead of a polished table/cards view |
- README is now being realigned to actual live state |
||||||
- Failed jobs do not yet surface log_excerpt nicely in UI |
- Global video jobs page should be fully wired into UI navigation and polished |
||||||
- No direct preview/download button for completed outputs in workshop |
- Dashboard template still contains some mixed button class styles that should be normalized |
||||||
- No health/storage/GPU dashboard panel yet |
- Health page can be expanded with per-processor breakdown later |
||||||
- No explicit processor chooser in UI |
- Processing metrics can be refined further into Intel/AMD/CPU buckets if desired |
||||||
- Output placement may later deserve a dedicated derived/video output area |
- Output browsing UX can still be improved further with richer previewing and filtering |
||||||
- Existing patch helper scripts were moved out of repo to keep git clean |
|
||||||
|
|
||||||
## Recommended Next Step |
## Recommended Next Step |
||||||
Proceed to alpha3-b: |
Proceed after alpha4 with: |
||||||
- replace raw JSON jobs output with styled job cards/table |
1. global video jobs page polish and filters |
||||||
- add output links for completed jobs |
2. per-processor GPU metrics split (Intel / AMD / CPU) |
||||||
- add visible failure details from log_excerpt |
3. scheduler documentation and/or scheduler UI visibility |
||||||
- add storage/GPU/worker health panel |
4. processed output browsing improvements in device view |
||||||
|
|||||||
@ -0,0 +1,51 @@ |
|||||||
|
{% extends "portal_base.html" %} |
||||||
|
|
||||||
|
{% block title %}OTB Cloud Health{% endblock %} |
||||||
|
|
||||||
|
{% block portal_content %} |
||||||
|
<div style="max-width:1100px;margin:0 auto;"> |
||||||
|
<div class="portal-page-header"> |
||||||
|
<div> |
||||||
|
<h1 class="portal-page-title">OTB Cloud Health</h1> |
||||||
|
<p class="portal-page-subtitle">Storage, processing, and usage overview.</p> |
||||||
|
</div> |
||||||
|
<div class="portal-toolbar" style="display:flex;gap:10px;flex-wrap:wrap;"> |
||||||
|
<a class="portal-btn" href="/dashboard">Back to Dashboard</a> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-list" style="display:flex;flex-direction:column;gap:14px;margin-top:18px;"> |
||||||
|
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||||
|
<h2>Uploads</h2> |
||||||
|
<p>Files: <strong>{{ uploaded_count }}</strong></p> |
||||||
|
<p>Space used: <strong>{{ uploaded_bytes }}</strong></p> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||||
|
<h2>LTS Storage</h2> |
||||||
|
<p>Files: <strong>{{ lts_count }}</strong></p> |
||||||
|
<p>Space used: <strong>{{ lts_bytes }}</strong></p> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||||
|
<h2>Archive Storage</h2> |
||||||
|
<p>Files: <strong>{{ archive_count }}</strong></p> |
||||||
|
<p>Space used: <strong>{{ archive_bytes }}</strong></p> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||||
|
<h2>Processing</h2> |
||||||
|
<p>Total jobs: <strong>{{ total_jobs }}</strong></p> |
||||||
|
<p>Completed jobs: <strong>{{ complete_jobs }}</strong></p> |
||||||
|
<p>Failed jobs: <strong>{{ failed_jobs }}</strong></p> |
||||||
|
<p>Total GPU time used: <strong>{{ gpu_time }}</strong></p> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||||
|
<h2>Disk Usage</h2> |
||||||
|
<p>Total tenant storage used: <strong>{{ total_used }}</strong></p> |
||||||
|
<p>Storage costs: <strong>placeholder</strong></p> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
{% endblock %} |
||||||
@ -0,0 +1,134 @@ |
|||||||
|
{% extends "portal_base.html" %} |
||||||
|
|
||||||
|
{% block title %}Global Video Jobs - OTB Cloud{% endblock %} |
||||||
|
|
||||||
|
{% block portal_content %} |
||||||
|
<style> |
||||||
|
.jobs-wrap { max-width: 1200px; margin: 0 auto; } |
||||||
|
.job-list { display:flex; flex-direction:column; gap:14px; margin-top:14px; } |
||||||
|
.job-card { background:rgba(255,255,255,0.04); border:1px solid rgba(255,255,255,0.08); border-radius:14px; padding:14px; } |
||||||
|
.job-head { display:flex; justify-content:space-between; align-items:center; gap:12px; flex-wrap:wrap; margin-bottom:8px; } |
||||||
|
.job-file { font-weight:700; word-break:break-word; } |
||||||
|
.job-head-right { display:flex; align-items:center; gap:8px; flex-wrap:wrap; } |
||||||
|
.job-badge { display:inline-block; padding:4px 10px; border-radius:999px; font-size:0.85rem; font-weight:700; border:1px solid rgba(255,255,255,0.14); } |
||||||
|
.job-badge.queued { background:rgba(245,158,11,0.18); } |
||||||
|
.job-badge.processing { background:rgba(59,130,246,0.18); } |
||||||
|
.job-badge.complete { background:rgba(34,197,94,0.18); } |
||||||
|
.job-badge.failed { background:rgba(239,68,68,0.18); } |
||||||
|
.job-grid { display:grid; grid-template-columns:repeat(auto-fit,minmax(180px,1fr)); gap:10px; margin-top:8px; } |
||||||
|
.job-meta { background:rgba(255,255,255,0.03); border-radius:10px; padding:10px; } |
||||||
|
.job-label { font-size:0.8rem; opacity:0.75; margin-bottom:4px; } |
||||||
|
.job-value { word-break:break-word; } |
||||||
|
.job-path, .job-error { margin-top:10px; padding:10px; border-radius:10px; background:rgba(255,255,255,0.03); word-break:break-word; } |
||||||
|
.job-error { border:1px solid rgba(239,68,68,0.22); } |
||||||
|
.job-empty { padding:16px; border-radius:12px; background:rgba(255,255,255,0.04); opacity:0.85; } |
||||||
|
.filter-note { opacity:0.8; margin-top:6px; } |
||||||
|
</style> |
||||||
|
|
||||||
|
<div class="jobs-wrap"> |
||||||
|
<div class="portal-page-header"> |
||||||
|
<div> |
||||||
|
<h1 class="portal-page-title">Global Video Jobs</h1> |
||||||
|
<p class="portal-page-subtitle">All video processing jobs across your devices.</p> |
||||||
|
<div class="filter-note">Newest first. Use this as a cross-device overview.</div> |
||||||
|
</div> |
||||||
|
<div class="portal-toolbar" style="display:flex;gap:10px;flex-wrap:wrap;"> |
||||||
|
<a class="portal-btn" href="/dashboard">Back to Dashboard</a> |
||||||
|
<a class="portal-btn" href="/health">Health</a> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
|
||||||
|
{% if jobs and jobs|length > 0 %} |
||||||
|
<div class="job-list"> |
||||||
|
{% for j in jobs %} |
||||||
|
<div class="job-card"> |
||||||
|
<div class="job-head"> |
||||||
|
<div class="job-file">{{ j.filename }}</div> |
||||||
|
<div class="job-head-right"> |
||||||
|
{% if j.output_relative_path and j.status == 'complete' %} |
||||||
|
<a class="portal-btn" href="/video-output/{{ j.id }}/view" target="_blank">View</a> |
||||||
|
<button class="portal-btn" type="button" onclick="sendToLTS({{ j.id }})">Send to LTS</button> |
||||||
|
<a class="portal-btn" href="/video-output/{{ j.id }}/download">Download Output</a> |
||||||
|
{% endif %} |
||||||
|
{% if j.status in ['failed', 'complete'] %} |
||||||
|
<button class="portal-btn" type="button" onclick="deleteJob({{ j.id }})">Delete</button> |
||||||
|
{% endif %} |
||||||
|
<span class="job-badge {{ j.status|lower }}">{{ j.status|upper }}</span> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
|
||||||
|
<div class="job-grid"> |
||||||
|
<div class="job-meta"><div class="job-label">Job ID</div><div class="job-value">{{ j.id }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Device</div><div class="job-value">{{ j.device_name }} ({{ j.device_id }})</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Profile</div><div class="job-value">{{ j.profile }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Rotation Override</div><div class="job-value">{{ j.rotation_override or 'auto' }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Processor</div><div class="job-value">{{ j.assigned_processor or 'pending' }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Progress</div><div class="job-value">{{ j.progress_percent }}%</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Created</div><div class="job-value">{{ j.created_at }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Started</div><div class="job-value">{{ j.started_at }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Completed</div><div class="job-value">{{ j.completed_at }}</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Original Size</div><div class="job-value" data-bytes="{{ j.original_size or '' }}"></div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Processed Size</div><div class="job-value" data-bytes="{{ j.processed_size or '' }}"></div></div> |
||||||
|
<div class="job-meta"><div class="job-label">GPU Time</div><div class="job-value">{{ j.gpu_seconds }}s</div></div> |
||||||
|
<div class="job-meta"><div class="job-label">Source File ID</div><div class="job-value">{{ j.source_file_id or '' }}</div></div> |
||||||
|
</div> |
||||||
|
|
||||||
|
{% if j.output_relative_path %} |
||||||
|
<div class="job-path"> |
||||||
|
<div class="job-label">Output</div> |
||||||
|
<div class="job-value">{{ j.output_relative_path }}</div> |
||||||
|
</div> |
||||||
|
{% endif %} |
||||||
|
|
||||||
|
{% if j.error_message %} |
||||||
|
<div class="job-error"> |
||||||
|
<div class="job-label">Error</div> |
||||||
|
<div class="job-value">{{ j.error_message }}</div> |
||||||
|
</div> |
||||||
|
{% endif %} |
||||||
|
</div> |
||||||
|
{% endfor %} |
||||||
|
</div> |
||||||
|
{% else %} |
||||||
|
<div class="job-empty">No video jobs yet.</div> |
||||||
|
{% endif %} |
||||||
|
</div> |
||||||
|
|
||||||
|
<script> |
||||||
|
function fmtBytes(v){ |
||||||
|
if(v === null || v === undefined || v === "") return ""; |
||||||
|
const n = Number(v); |
||||||
|
if(!Number.isFinite(n)) return String(v); |
||||||
|
if(n < 1024) return `${n} B`; |
||||||
|
if(n < 1024*1024) return `${(n/1024).toFixed(1)} KB`; |
||||||
|
if(n < 1024*1024*1024) return `${(n/1024/1024).toFixed(2)} MB`; |
||||||
|
return `${(n/1024/1024/1024).toFixed(2)} GB`; |
||||||
|
} |
||||||
|
|
||||||
|
document.querySelectorAll("[data-bytes]").forEach(el => { |
||||||
|
el.textContent = fmtBytes(el.getAttribute("data-bytes")); |
||||||
|
}); |
||||||
|
|
||||||
|
async function deleteJob(jobId){ |
||||||
|
if(!confirm("Delete this job from the list?")) return; |
||||||
|
const r = await fetch(`/api/video/jobs/${jobId}/delete`, { method: "POST" }); |
||||||
|
if(r.ok){ |
||||||
|
window.location.reload(); |
||||||
|
} else { |
||||||
|
const text = await r.text(); |
||||||
|
alert(text || "Delete failed"); |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
async function sendToLTS(jobId){ |
||||||
|
if(!confirm("Send this output to LTS storage?")) return; |
||||||
|
const r = await fetch(`/video-output/${jobId}/send-to-lts`, { method: "POST" }); |
||||||
|
if(r.ok){ |
||||||
|
window.location.reload(); |
||||||
|
} else { |
||||||
|
const text = await r.text(); |
||||||
|
alert(text || "Send to LTS failed"); |
||||||
|
} |
||||||
|
} |
||||||
|
</script> |
||||||
|
{% endblock %} |
||||||
Loading…
Reference in new issue