From 89e35cf42ec6f679dd5849aece5f9f8e626b8653 Mon Sep 17 00:00:00 2001 From: Joanna White <37188631+digitensions@users.noreply.github.com> Date: Fri, 12 Apr 2024 15:09:49 +0100 Subject: [PATCH] Update Case_study.md --- Doc/Case_study.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/Case_study.md b/Doc/Case_study.md index 4a43c96d..4018b172 100644 --- a/Doc/Case_study.md +++ b/Doc/Case_study.md @@ -217,9 +217,9 @@ It decodes the FFV1 Matroska back to image sequence, checks the logs for ```Reve We began using RAWcooked to convert 3 petabytes of 2K DPX sequence data to FFV1 Matroska for our *Unlocking Film Heritage* project. This lossless compression to FFV1 has saved us an estimated 1600TB of storage space, which has saved thousands of pounds of additional magnetic storage tape purchases. Undoubtedly this software offers amazing financial incentives with all the benefits of open standards and open-source tools. It also creates a viewable video file of an otherwise invisible DPX scan, so useful for viewing the unseen technology of film. We plan to begin testing RAWcooked encoding of TIFF image sequences shortly with the intention of moving DCDM image sequences to FFV1. Today, our workflow runs 24/7 performing automated encoding of business-as-usual DPX sequences with relatively little overview. There is a need for manual intervention when repeated errors are encountered. This is usually indicated when an image sequences doesn't make it to our Digital Preservation Infrastructure. Most often this is caused by a new image sequence 'flavour' that we do not have covered by our RAWcooked licence, or sometimes it can indicate a problem with either RAWcooked or FFmpeg file encoding a specific DPX scan - there can be many differences found in DPX metadata depending on the scanning technology. Where errors are found by our automations these are reported to an error log named after the image seqeuence, a build up of reported errors will indicate repeated problems. -In recent years we have been encoding a mix of 2K and 4K of various bit depths, seeing our licence expand. When we solely encoded 2K sequences we found we could run multiple parallel processes with good efficiency, seeing 32 concurrent encodings running at once. This was before we implemented the '--all' command which calculates checksums adding them to the reversibility data and runs a checksum comparison of the Matroska after encoding has completed which expands the encoding process. We saw our concurrency drop to accomodate the more detailed encoding process, particularly as our workflow introduced a final '--check' pass against the Matroska file that automated the deletion of the DPX sequence when successful. +In recent years we have been encoding a mix of 2K and 4K of various bit depths, seeing our licence expand. When we solely encoded 2K sequences we found we could run multiple parallel processes with good efficiency, seeing 32 concurrent encodings running at once. This was before we implemented the ```--all``` command which calculates checksums adding them to the reversibility data and runs a checksum comparison of the Matroska after encoding has completed which expands the encoding process. We saw our concurrency drop to accomodate this more detailed encoding process, particularly as our workflow introduced a final ```--check``` pass against the Matroska file that automated the deletion of the DPX sequence, when successful. -Since running an increasing number of 4K sequences we find we have better '--all' encoding and parallel '--check' efficiency running just two parallel encodings at any given time. We recently ran a review of our 4K and 2K encoding timings. Below are some recent 4K DPX encoding times using RAWcooked's '--all' command with a maximum of two parallel encodings, and where we can assume another single '--check' run is underway from the server: +Since running an increasing number of 4K sequences we find we have better ```--all``` encoding and parallel ```--check``` efficiency running just two parallel encodings at any given time. We recently ran a review of our 4K and 2K encoding timings. Below are some recent 4K DPX encoding times using RAWcooked's ```--all``` command with a maximum of two parallel encodings, and where we can assume another single ```--check``` run is underway from the server: * Parallel 4K RGB 16-bit DPX (699.4 GB) - MKV duration 5:10 (639.8 GB) - encoding time 5:17:00 - MKV 8.5% smaller than DPX * Parallel 4K RGB 16-bit DPX (723.1 GB) - MKV duration 5:20 (648.9 GB) - encoding time 5:40:07 - MKV 10.25% smaller than DPX @@ -230,7 +230,7 @@ Since running an increasing number of 4K sequences we find we have better '--all * Parallel 4K RGB 12-bit DPX (887.3 GB) - MKV duration 10:54 (208.7 GB) - encoding time 5:02:00 - MKV 76.5% smaller than DPX ** ** Where the MKV is significantly smaller than the DPX then a black and while filter will have been applied to an RGB scan, as in these cases. -A separate 2K solo and parallel encoding test revealed much quicker encoding times for >10 minute sequences, again using the '--all' command and where we can assume another single '--check' run is underway: +A separate 2K solo and parallel encoding test revealed much quicker encoding times for >10 minute sequences, again using the ```--all``` command and where we can assume another single ```--check``` run is also working in parallel: * Solo 2K RGB 12-bit DPX (341 GB) - MKV duration 16:16 - encoding time 1:20:00 - MKV 22.5% smaller than DPX * Solo 2K RGB 16-bit DPX (126 GB) - MKV duration 11:42 - encoding time 1:02:00 - MKV was 30.6% smaller than the DPX