sqlite3 storage implementation

[smartin015]

Storage:

• Establish base pewee DB models and queries
• Switch to edit/commit job flow
• Improved set reduction calculation
• Add a Version model to the DB for future mutation handling
• Implement legacy JSON import

UX

• Switch to single tab with subtabs
• Allow reordering queues in settings page
• Hide queue titles and the send button when no network queues
• Left/right align history headers
• Eliminate refresh button
• Add description of strategies in settings page
• Implement job deletions
• add queue names (local vs lan queues)
• Display number of printers connected to lan queues
• Connected peers show up in the UI
• Add tooltips to all buttons
• Show remote printers working on prints (can test with shell.py)
• Refresh UI when job submitted appears in local queue
• Remove selection check boxes for sets and replace with trash can icon (when in edit mode)
• Show hint for network queues when no network queues
• Fix UI updates (peer added/removed/state change, job sent to remote queue & received)
• Prevent edits to active/acquired jobs
• Auto update history as prints complete
• Fix progress bar animations when adjusting counts in edit mode
• Fix update spam when initializing LAN queues
• New LAN queue job should appear when it's added by a peer
• Select and Print options should be disabled on .gjobs in File manager
• Selection boxes don't revert when state updated
• Move selects to left side (to right of drag handle)
• Saving job with a set that was moved out of it should not persist that set

LAN behavior

• Setup of server / additing & removing LAN queues
• Job submission to LAN queue
• Observe transmission of job to peer
• Refactor supervisor code to handle multiple queues
• Unit test supervisor code, queries, etc.
• Complete documentation
• Fix Travis CI builds
• Implement file hosting/fetching by hash and peer
• Integration test for LAN queues that tests multi-peer interaction
• Implement non-local set path resolution using new file hosting
• Successful run-to-completion of host-issued LAN queue job
• reduce file copies when submitting a job / posting to fileshare
• Create a LAN queue direct job-upload script for testing
• Successful run-to-completion of remote-issued LAN queue job
• Fix "failed to store full dump" warning spam
• Possible failure to initialize pysyncobj values if previously present peer not present?
• Humanized network names
• Fix lan job deletion - it comes back next update

Job Files

• Base pack/unpack library
• Include version in job manifest
• Export queue jobs to file panel
• Clicking "add" unpacks the job and loads into the local queue

Queue behavior

• Integration test framework
• Confirm correct queue behavior when printing sets/jobs with count > 1
• Confirm profile restriction successfully restricts the jobs/sets that a printer can handle
• Fix permissions checks on API calls
• Disallow selecting running jobs
• Release job / end run when driver aborts from printing

[smartin015]

Bugs

• Queue reorder should cause immediate UI change
• Active set not shown in local queue until refresh
• Files list requires reload after exporting a job
• "Acquired by Foo" text looks out of place in UX - use an icon / background color change instead?
• Hover edit div still shown (just without icon) when job is acquired
• Job refresh requires page reload to show change (need to return state)
• "From" queue shouldn't be listed in job sending dropdown
• Compatible printer profiles for set should be visible without needing expansion
• Print profile not saved from settings, causes failure to match profiled sets
• Saving job with dragged-in set after it was deleted from dragged-out set makes the dragged-in set disappear
• Fix peerprint unit tests & re-deploy to fix CPQ unit tests

Basic QA check:

• Adding queue
• Removing queue
• Reordering queues
• Renaming queues / changing address
• Cancel settings without reorder
• Add file with no jobs
• Add file with editable job
• Add file with no editable jobs
• Add file while printing
• Repeat file add
• Drag set between draft jobs
• Save job with dragged-in set, dragged-out set (+ refresh)
• Drag reorder jobs (first to last, last to first, middle)
• Bulk delete job
• Bulk refresh job
• Submit job to LAN queue
• Delete job from LAN queue
• Export job
• Import job from file list
• Attempt print of job with non-generic profile match

Cleanup:

• Consistent JS API
• Split off route handlers from base plugin class
• Move all static data to data/__init__.py
• Split off script running part of base plugin into script_runner.py
• Upgrade to octoprint v1.8.0 and confirm everything still works

[smartin015]

UX jam notes

Style changes

• Drag zones are odd/fiddly, should be expanded a bit
• Click spot on edit button limited to the icon - should be whole thing
• Spacing
  • Top button spacing (add job / start managing) need spacing equiavlent to what's in the State panel
  • Padding is a bit close on the top multi-select section - need to standardize paddings, also on bottom of queueset and between jobs
• Icons
  • Perforated dots for drag handles (grip-vertical)
  • history - display success as standard, but call-out failures with prefix icons
  • .gjob file needs a bit more clarity - hide the open/print and additional data dropdown buttons rather than disabling them. Also maybe add file-archive icon as far.
  • Change "export" icon to something visually different from "send" (e.g. a download icon)
• Make draft jobs more obvious
  • hide progress bars and let names flow to full width
  • Thicken border?
  • Taller top row?
  • blue save button
  • Move job drag handle further left (less padding), appear on hover
• Dynamically display extra bits of Bed Cooldown settings based on whether they're enabled. Also need a better description on "why would I use this?"
• Change "save job" button to be a button at the bottom of the job, instead of a hover.
• Remove sliver of green in progress bars of unprinted sete
• Add a header to printer profiles in set
• Split selection button group away from checkbox button, with standard spacing
• Queue network name textbox is formatted weirdly, should be wider.

Content changes

• Empty queue - add hint on how to add things in the open space.
• Remove "save to save, close to discard" message in settings Scripts page.
• Under settings Queues: "Follow this guide to configure your LAN queues...". Also lead with "Renaming a queue will cause..."
• "Queue finished script" --> "Finished script". For each, add a hover "info" icon that describes them further.
• Bug: show a top-level warning bar if some spools don't have materials assigned to them, to prevent later confusion
• Documentation "Use Jobs to group" -> "Use Sets and Jobs to group". Also documentation needs to mention adding spools before trying to select a material

Behavior changes

• Validation on queue addition - ensure there's an address provided. Also add a placeholder on input
• Add behavior on draft job not immediately clear
• Shift click bulk selection
• Drag job over to LAN queue to submit it, and eliminate send button
• Prevent dragging editable jobs
• Bug: material selection not saving into job. Also need to hide material selection when job not in edit mode.
• By default, show a "send to queue" modal with confirmation unless sets are strictly profiled.
  • Add a default warning - once submitted, jobs cannot be edited or transferred back.
• Throw validation error when submitting job that isn't compatible with a network queue.
• Throw validation warning when submitting job without specified profiles into a heterogeneneous queue.
• Bug: material selection claims to be enabled in settings even though spoolmanager not installed.
• Bug: Edits get clobbered on UI refreshes
• Bug: ABC job order move to BAC causes BCA
• Adding set to draft job, then cancelling causes added set to persist

[smartin015]

UX feedback #2

Content

• Dialog warning "to the lan queue" -> "to a lan queue"
• change "0 peers" -> "1 printer"
• For spool material missing warning - indicate some way how to fix it
• "drag a queue item here" -> "drag in a set from a different edited job"
• In history, "started" -> "in progress"
• Have blank option in materials dropdowns to be "Any"
• Remove "Extruder" entries in set
• Call out in submit dialog: "all progress will be reset"
• Split out hint for material selection: "install spoolmanager" vs "add spools and reload"
• title on "connecting" status to describe what's happening
• Publish the doc guide on network queues

Style

• Drag area into empty queue should be wider (doesn't respond when dragging into left side)
• LAN queue delete icon thinks it's part of a group
• Multiselect should be on the left side, over the other checkboxes
• Highlight drop zones for draggable item when drag starts
• Icon for mismatch profile - change to something to indicate skippiness
• Profiles dropdown - move to the right of the profile badges, in line with them
• Queue status message should be more attention grabbing when manual intervention is needed (e.g. waiting for correct filament). Also pause the progress bar animations

Behavior

• Default job names on creation: "job 1, job 2" etc.
• Pressing "enter" on any draft job input should save the job
• Better file save output: change timestamp out for number suffix, remove cpq_ prefix
• "Start managing" seems to put queues in "Connecting..." status
• Print queue state text says "Inactive" even when actively printing/managing
• Job submitted to queue claims to be almost complete (with no sets completed). Confirm "remaining" is set to "count" on submission
• Spool manager enabled --> still shows note to enable spoolmanager
• Block settings submission when lan queue not valid
• Complete/abort started jobs in history on server startup
• Clicking "stop managing" while printing, then "start managing" before it terminates causes inconsistency
• Jobs sometimes don't disappear from local queue when adding to LAN queue (repro may be difficult)
• Next_set should check for archive queue membership
• Release current print job and acquire earlier job if there is one - don't print a whole job to completion.
• Printing from an unpacked job constantly reprints the same file (considered 'aborted')

Wontfix:

• Enter should move to next thing to edit

[smartin015]

Physical testing:

Behavior

• Peer address is reported as 0.0.0.0:7890, not <real_ip:7890> - need to somehow resolve this.
• Peer address 0.0.0.0 issue also applies to the fileshare - causes failure to fetch.
• Add an "Idle" state which has queue still active, but waiting for LAN jobs to arrive
• Shorten acquire lock to ~1 minute
• Troubleshoot: delta printer failed to start profile-selected job on network queue
• LAN queue shows print acquired, but no indication that another printer is printing it. enable progress based on active set (pass in peer status?)
• Include printer profile in peer status hovertext (to assess profile matching)
• Confirm fix - updating spaghetti timeout / max retries should not raise exception due to string (TypeError when in _state_failure checking retries)
• Adjust set/job decrement logic to include a profile, reduce job remaining even if incompatible sets not fully finished.
• Release job when going idle (Rocinante held onto a job with an extra set with nonmatching profile)
• Refreshing / Saving jobs should deselect them
• Limit history to most recent 100 prints
• Unpacking job fails with Permission denied on os.makedirs in queues/local.py l106
• Add unit test confirming sequential "next set" behavior.

Style

• Adjust css to be compatible with Themeify (e.g. jobs in edit mode are weird when viewed in dark theme)
• Bring multiselect box down, closer to queue list
• Misaligned checkbox on LAN queues (because of hidden grip handles?)

Need to find repro case:

• Queue fails to pick up new job after all other jobs finished (on Rocinante)
• ghost job left over after submission happened again :-\

Enhancements

• Remove greyed out dropdown on .gjob file in files list
• Flyout status indication for all printers in the queue (instead of hovertext).

Video Notes

• Microphone too close to keyboard/mouse, next time try to isolate. Also reduce breathing
• Next time, have a specific use case (e.g. box with lid)
• Add "github feedback" in description area
• Lengthen section bars, (pause audio?)