11 changed files with 1506 additions and 277 deletions
@ -1,51 +1,85 @@
|
||||
# PROJECT_STATE.md |
||||
|
||||
Project: OTB Cloud |
||||
Version: v1.1.0-alpha3 |
||||
Updated: 2026-04-19 |
||||
Version: v1.1.0-alpha4 |
||||
Updated: 2026-04-20 |
||||
Location: /opt/otb_cloud |
||||
|
||||
## Current State |
||||
OTB Cloud now has a functioning workshop-driven video processing pipeline. |
||||
OTB Cloud now has a working multi-profile, multi-GPU video processing pipeline integrated into the tenant storage platform. |
||||
|
||||
### Confirmed Working |
||||
- Portal and branded UI shell |
||||
- Device browser |
||||
- File selection flow into Video Workshop |
||||
- Portal-branded OTB Cloud dashboard |
||||
- Device creation and browsing |
||||
- File-ID based workshop staging |
||||
- Device-specific Video Workspace access from dashboard |
||||
- Video Workshop page |
||||
- Enqueue API |
||||
- Jobs API |
||||
- MariaDB-backed video_jobs integration |
||||
- Tenant/device path resolution for queued jobs |
||||
- Worker service startup and queue pickup |
||||
- Worker-side absolute path resolution from tenant storage_root |
||||
- Intel iGPU processing path |
||||
- Successful completed output for device 27 (ripper) |
||||
|
||||
### Latest Proven Result |
||||
A queued workshop job for: |
||||
- source file: 05142013003.mp4 |
||||
- device: 27 (ripper) |
||||
|
||||
completed successfully with: |
||||
- assigned_processor: intel |
||||
- status: complete |
||||
- progress_percent: 100 |
||||
- output_relative_path: |
||||
devices/ripper/originals/20260413T210325474049Z__05142013003_processed.mp4 |
||||
- Multi-profile processing selection: |
||||
- default |
||||
- compress |
||||
- hq |
||||
- Manual rotation override option: |
||||
- auto/default behavior when unchecked |
||||
- selectable 90 / 180 / 270 override when enabled |
||||
- Job queue API and job listing API |
||||
- File-ID based source resolution |
||||
- Output routing to: |
||||
- devices/<device>/video/ |
||||
- Profile-specific output filenames |
||||
- Completed job actions: |
||||
- View |
||||
- Send to LTS |
||||
- Download Output |
||||
- Delete |
||||
- Failed job delete action |
||||
- LTS routing by file type: |
||||
- lts/video |
||||
- lts/archived |
||||
- lts/pictures |
||||
- Health page |
||||
- Lifetime processing metrics retained after visible job deletion |
||||
- Intel + AMD GPU processing both in service |
||||
- GPU time accounting active in Health page |
||||
- Global video jobs route exists in codebase |
||||
- Processed video section exists in device browser flow |
||||
|
||||
### Processing / GPU Behavior |
||||
Current live behavior: |
||||
- both GPUs take jobs |
||||
- AMD prioritizes heavier / HQ work first |
||||
- Intel handles lighter work |
||||
- workers continue taking suitable jobs from the queue batch as available |
||||
|
||||
### Latest Proven Health State |
||||
Health page currently shows stable cumulative values including: |
||||
- uploaded file counts and space |
||||
- LTS counts and space |
||||
- archive counts and space |
||||
- total jobs |
||||
- completed jobs |
||||
- failed jobs |
||||
- cumulative GPU time that does not zero out when workshop cards are deleted |
||||
|
||||
### Current Storage Layout |
||||
- originals remain in device originals tree |
||||
- processed outputs go to: |
||||
- devices/<device>/video/ |
||||
- LTS destinations include: |
||||
- lts/video/ |
||||
- lts/archived/ |
||||
- lts/pictures/ |
||||
|
||||
## Known Remaining Improvements |
||||
- Jobs panel is still raw JSON instead of a polished table/cards view |
||||
- Failed jobs do not yet surface log_excerpt nicely in UI |
||||
- No direct preview/download button for completed outputs in workshop |
||||
- No health/storage/GPU dashboard panel yet |
||||
- No explicit processor chooser in UI |
||||
- Output placement may later deserve a dedicated derived/video output area |
||||
- Existing patch helper scripts were moved out of repo to keep git clean |
||||
- README is now being realigned to actual live state |
||||
- Global video jobs page should be fully wired into UI navigation and polished |
||||
- Dashboard template still contains some mixed button class styles that should be normalized |
||||
- Health page can be expanded with per-processor breakdown later |
||||
- Processing metrics can be refined further into Intel/AMD/CPU buckets if desired |
||||
- Output browsing UX can still be improved further with richer previewing and filtering |
||||
|
||||
## Recommended Next Step |
||||
Proceed to alpha3-b: |
||||
- replace raw JSON jobs output with styled job cards/table |
||||
- add output links for completed jobs |
||||
- add visible failure details from log_excerpt |
||||
- add storage/GPU/worker health panel |
||||
Proceed after alpha4 with: |
||||
1. global video jobs page polish and filters |
||||
2. per-processor GPU metrics split (Intel / AMD / CPU) |
||||
3. scheduler documentation and/or scheduler UI visibility |
||||
4. processed output browsing improvements in device view |
||||
|
||||
@ -0,0 +1,51 @@
|
||||
{% extends "portal_base.html" %} |
||||
|
||||
{% block title %}OTB Cloud Health{% endblock %} |
||||
|
||||
{% block portal_content %} |
||||
<div style="max-width:1100px;margin:0 auto;"> |
||||
<div class="portal-page-header"> |
||||
<div> |
||||
<h1 class="portal-page-title">OTB Cloud Health</h1> |
||||
<p class="portal-page-subtitle">Storage, processing, and usage overview.</p> |
||||
</div> |
||||
<div class="portal-toolbar" style="display:flex;gap:10px;flex-wrap:wrap;"> |
||||
<a class="portal-btn" href="/dashboard">Back to Dashboard</a> |
||||
</div> |
||||
</div> |
||||
|
||||
<div class="job-list" style="display:flex;flex-direction:column;gap:14px;margin-top:18px;"> |
||||
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||
<h2>Uploads</h2> |
||||
<p>Files: <strong>{{ uploaded_count }}</strong></p> |
||||
<p>Space used: <strong>{{ uploaded_bytes }}</strong></p> |
||||
</div> |
||||
|
||||
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||
<h2>LTS Storage</h2> |
||||
<p>Files: <strong>{{ lts_count }}</strong></p> |
||||
<p>Space used: <strong>{{ lts_bytes }}</strong></p> |
||||
</div> |
||||
|
||||
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||
<h2>Archive Storage</h2> |
||||
<p>Files: <strong>{{ archive_count }}</strong></p> |
||||
<p>Space used: <strong>{{ archive_bytes }}</strong></p> |
||||
</div> |
||||
|
||||
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||
<h2>Processing</h2> |
||||
<p>Total jobs: <strong>{{ total_jobs }}</strong></p> |
||||
<p>Completed jobs: <strong>{{ complete_jobs }}</strong></p> |
||||
<p>Failed jobs: <strong>{{ failed_jobs }}</strong></p> |
||||
<p>Total GPU time used: <strong>{{ gpu_time }}</strong></p> |
||||
</div> |
||||
|
||||
<div class="job-card" style="background:rgba(255,255,255,0.04);padding:14px;border-radius:14px;"> |
||||
<h2>Disk Usage</h2> |
||||
<p>Total tenant storage used: <strong>{{ total_used }}</strong></p> |
||||
<p>Storage costs: <strong>placeholder</strong></p> |
||||
</div> |
||||
</div> |
||||
</div> |
||||
{% endblock %} |
||||
@ -0,0 +1,134 @@
|
||||
{% extends "portal_base.html" %} |
||||
|
||||
{% block title %}Global Video Jobs - OTB Cloud{% endblock %} |
||||
|
||||
{% block portal_content %} |
||||
<style> |
||||
.jobs-wrap { max-width: 1200px; margin: 0 auto; } |
||||
.job-list { display:flex; flex-direction:column; gap:14px; margin-top:14px; } |
||||
.job-card { background:rgba(255,255,255,0.04); border:1px solid rgba(255,255,255,0.08); border-radius:14px; padding:14px; } |
||||
.job-head { display:flex; justify-content:space-between; align-items:center; gap:12px; flex-wrap:wrap; margin-bottom:8px; } |
||||
.job-file { font-weight:700; word-break:break-word; } |
||||
.job-head-right { display:flex; align-items:center; gap:8px; flex-wrap:wrap; } |
||||
.job-badge { display:inline-block; padding:4px 10px; border-radius:999px; font-size:0.85rem; font-weight:700; border:1px solid rgba(255,255,255,0.14); } |
||||
.job-badge.queued { background:rgba(245,158,11,0.18); } |
||||
.job-badge.processing { background:rgba(59,130,246,0.18); } |
||||
.job-badge.complete { background:rgba(34,197,94,0.18); } |
||||
.job-badge.failed { background:rgba(239,68,68,0.18); } |
||||
.job-grid { display:grid; grid-template-columns:repeat(auto-fit,minmax(180px,1fr)); gap:10px; margin-top:8px; } |
||||
.job-meta { background:rgba(255,255,255,0.03); border-radius:10px; padding:10px; } |
||||
.job-label { font-size:0.8rem; opacity:0.75; margin-bottom:4px; } |
||||
.job-value { word-break:break-word; } |
||||
.job-path, .job-error { margin-top:10px; padding:10px; border-radius:10px; background:rgba(255,255,255,0.03); word-break:break-word; } |
||||
.job-error { border:1px solid rgba(239,68,68,0.22); } |
||||
.job-empty { padding:16px; border-radius:12px; background:rgba(255,255,255,0.04); opacity:0.85; } |
||||
.filter-note { opacity:0.8; margin-top:6px; } |
||||
</style> |
||||
|
||||
<div class="jobs-wrap"> |
||||
<div class="portal-page-header"> |
||||
<div> |
||||
<h1 class="portal-page-title">Global Video Jobs</h1> |
||||
<p class="portal-page-subtitle">All video processing jobs across your devices.</p> |
||||
<div class="filter-note">Newest first. Use this as a cross-device overview.</div> |
||||
</div> |
||||
<div class="portal-toolbar" style="display:flex;gap:10px;flex-wrap:wrap;"> |
||||
<a class="portal-btn" href="/dashboard">Back to Dashboard</a> |
||||
<a class="portal-btn" href="/health">Health</a> |
||||
</div> |
||||
</div> |
||||
|
||||
{% if jobs and jobs|length > 0 %} |
||||
<div class="job-list"> |
||||
{% for j in jobs %} |
||||
<div class="job-card"> |
||||
<div class="job-head"> |
||||
<div class="job-file">{{ j.filename }}</div> |
||||
<div class="job-head-right"> |
||||
{% if j.output_relative_path and j.status == 'complete' %} |
||||
<a class="portal-btn" href="/video-output/{{ j.id }}/view" target="_blank">View</a> |
||||
<button class="portal-btn" type="button" onclick="sendToLTS({{ j.id }})">Send to LTS</button> |
||||
<a class="portal-btn" href="/video-output/{{ j.id }}/download">Download Output</a> |
||||
{% endif %} |
||||
{% if j.status in ['failed', 'complete'] %} |
||||
<button class="portal-btn" type="button" onclick="deleteJob({{ j.id }})">Delete</button> |
||||
{% endif %} |
||||
<span class="job-badge {{ j.status|lower }}">{{ j.status|upper }}</span> |
||||
</div> |
||||
</div> |
||||
|
||||
<div class="job-grid"> |
||||
<div class="job-meta"><div class="job-label">Job ID</div><div class="job-value">{{ j.id }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Device</div><div class="job-value">{{ j.device_name }} ({{ j.device_id }})</div></div> |
||||
<div class="job-meta"><div class="job-label">Profile</div><div class="job-value">{{ j.profile }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Rotation Override</div><div class="job-value">{{ j.rotation_override or 'auto' }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Processor</div><div class="job-value">{{ j.assigned_processor or 'pending' }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Progress</div><div class="job-value">{{ j.progress_percent }}%</div></div> |
||||
<div class="job-meta"><div class="job-label">Created</div><div class="job-value">{{ j.created_at }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Started</div><div class="job-value">{{ j.started_at }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Completed</div><div class="job-value">{{ j.completed_at }}</div></div> |
||||
<div class="job-meta"><div class="job-label">Original Size</div><div class="job-value" data-bytes="{{ j.original_size or '' }}"></div></div> |
||||
<div class="job-meta"><div class="job-label">Processed Size</div><div class="job-value" data-bytes="{{ j.processed_size or '' }}"></div></div> |
||||
<div class="job-meta"><div class="job-label">GPU Time</div><div class="job-value">{{ j.gpu_seconds }}s</div></div> |
||||
<div class="job-meta"><div class="job-label">Source File ID</div><div class="job-value">{{ j.source_file_id or '' }}</div></div> |
||||
</div> |
||||
|
||||
{% if j.output_relative_path %} |
||||
<div class="job-path"> |
||||
<div class="job-label">Output</div> |
||||
<div class="job-value">{{ j.output_relative_path }}</div> |
||||
</div> |
||||
{% endif %} |
||||
|
||||
{% if j.error_message %} |
||||
<div class="job-error"> |
||||
<div class="job-label">Error</div> |
||||
<div class="job-value">{{ j.error_message }}</div> |
||||
</div> |
||||
{% endif %} |
||||
</div> |
||||
{% endfor %} |
||||
</div> |
||||
{% else %} |
||||
<div class="job-empty">No video jobs yet.</div> |
||||
{% endif %} |
||||
</div> |
||||
|
||||
<script> |
||||
function fmtBytes(v){ |
||||
if(v === null || v === undefined || v === "") return ""; |
||||
const n = Number(v); |
||||
if(!Number.isFinite(n)) return String(v); |
||||
if(n < 1024) return `${n} B`; |
||||
if(n < 1024*1024) return `${(n/1024).toFixed(1)} KB`; |
||||
if(n < 1024*1024*1024) return `${(n/1024/1024).toFixed(2)} MB`; |
||||
return `${(n/1024/1024/1024).toFixed(2)} GB`; |
||||
} |
||||
|
||||
document.querySelectorAll("[data-bytes]").forEach(el => { |
||||
el.textContent = fmtBytes(el.getAttribute("data-bytes")); |
||||
}); |
||||
|
||||
async function deleteJob(jobId){ |
||||
if(!confirm("Delete this job from the list?")) return; |
||||
const r = await fetch(`/api/video/jobs/${jobId}/delete`, { method: "POST" }); |
||||
if(r.ok){ |
||||
window.location.reload(); |
||||
} else { |
||||
const text = await r.text(); |
||||
alert(text || "Delete failed"); |
||||
} |
||||
} |
||||
|
||||
async function sendToLTS(jobId){ |
||||
if(!confirm("Send this output to LTS storage?")) return; |
||||
const r = await fetch(`/video-output/${jobId}/send-to-lts`, { method: "POST" }); |
||||
if(r.ok){ |
||||
window.location.reload(); |
||||
} else { |
||||
const text = await r.text(); |
||||
alert(text || "Send to LTS failed"); |
||||
} |
||||
} |
||||
</script> |
||||
{% endblock %} |
||||
Loading…
Reference in new issue