Plot Window Freezes and Videos Don't End at the Specified Time

[dankunk]

I have been having some issues running campy on my system. Specifically, when I run the campy-acquire ./configs/my_config.yaml command, the camera capture initiates, but the plot window is frozen and the videos won't end at the specified time from recTimeInSec:. I can keyboard interrupt the acquisition just fine; however, with say recTimeInSec: 30, the acquisition will continue past this point and will only stop after said keyboard interruption.

• I just installed the package per the instructions in the readme (conda with py3.7, pyspin wheel for version 3.7, etc.).

• The camera I am using is a FLIR BFS-U3-32s4M-C USB3.

• My computer configuration is a Dell Precision 5860 machine with: Intel(R) Xeon(R) w5-2545, 128GB RAM, and NVIDIA 4000 Ada.

• I tested this with a few of the example configs and the same behavior is present with CPU/GPU compression.

Here is an example config that I was using:

# This beginner-level config uses the CPU (gpuID: -1) to compress video
# using ffmpeg's X264 codec

# Recording parameters
videoFolder: "./test"
videoFilename: "test0.mp4"
frameRate: 60
recTimeInSec: 30

# Camera parameters
numCams: 1
cameraMake: "flir"
frameWidth: 2048
frameHeight: 1536

# Flir settings
cameraTrigger: "None" # "Line3"
bufferSize: 100
cameraExposureTimeInUs: 1500
cameraGain: 1
disableGamma: True

# Compression parameters
gpuID: -1
pixelFormatInput: "gray"
pixelFormatOutput: "rgb0"

# Display parameters
displayFrameRate: 10

And now the output:

(campy) PS C:\Users\nalamlab\campy> campy-acquire ./configs/campy_config_1cam_flir.yaml
\\?\C:\Users\nalamlab\AppData\Local\anaconda3\envs\campy\Scripts\campy-acquire-script.py:15: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import load_entry_point
Opened: ./test\Camera0\test0.mp4 using CPU to compress the stream.
Trigger source set to None...
ERROR:root:Caught error at cameras/flir.py ConfigureCustomImageSettings: Spinnaker: GenICam::AccessException= Node is not writable. : AccessException thrown in node 'BalanceWhiteAuto' while calling 'BalanceWhiteAuto.SetIntValue()' (file 'EnumerationT.h', line 83) [-2006]
Opened Camera0: flir Blackfly S BFS-U3-32S4M serial# 24382851
Camera0 ready to trigger.
SIGINT received. KeyboardInterrupt has been queued.
Closing Camera0... Please wait.
ERROR:root:Caught exception: list index out of range
Closing video writer for Camera0. Please wait...
SIGINT received. KeyboardInterrupt has been handled.
Closing systems...
Exiting campy...
(campy) PS C:\Users\nalamlab\campy>

As you can see, recording time for this acquisition should only be 30 seconds, but after waiting 2 minutes with no ending and the plotting window frozen, I just interrupted with control+c on the keyboard.

Here is a screenshot of the issue with the frozen plotting window:

And some properties of the test video saved to disc. Notice how the video is larger than 30 seconds. The video plays just fine, but it seems to be capturing more video than I want it to!

I'm going to keep trying to solve the issue and I will post any updates here.

I think I will also try the AV1 branch build as I have read that potentially the issues with the plotting window are fixed there.

ALSO, THANK YOU VERY MUCH FOR DEVELOPING THIS TOOL, IT'S AWESOME!!

[dankunk]

From what I can tell after a bit of diagnosing, the issue is stemming from this error:

ERROR:root:Caught error at cameras/flir.py ConfigureCustomImageSettings: Spinnaker: GenICam::AccessException= Node is not writable. : AccessException thrown in node 'BalanceWhiteAuto' while calling 'BalanceWhiteAuto.SetIntValue()' (file 'EnumerationT.h', line 83) [-2006]

From what I have read online and in the spinnaker docs, using white balance with a monochrome camera is null since there are no colors to balance.

Are there some different arguments that I should be including with my FLIR monochrome camera config to make sure that it is working correctly?

With the master build, the video can record just fine with this error being thrown (although plotting window freezes and acquisition needs to be stopped manually. In the AVI build, the camera initiates but the video is corrupted and I still need to stop video via keyboard.

I'm just running the AV1 build in a new conda envi called campy2, all configs are more or less the same.

Here is that output with the AV1 build:

(campy2) PS C:\Users\nalamlab\campy2> campy-acquire .\configs\campy_config_1cam_flir.yaml
Opened Video [CPU]: test\Camera0\Camera0_20241207_151606_0-2399.mp4
Loading settings from Default ...
Trigger source set to None...
ERROR:root:Caught error at cameras/flir.py ConfigureCustomImageSettings: Spinnaker: GenICam::AccessException= Node is not writable. : AccessException thrown in node 'BalanceWhiteAuto' while calling 'BalanceWhiteAuto.SetIntValue()' (file 'EnumerationT.h', line 83) [-2006]
Opened Camera0: flir Blackfly S BFS-U3-32S4M serial# 24382851
SIGINT received. KeyboardInterrupt has been queued.
WARNING:imageio_ffmpeg:No frames have been written; the written video is invalid.
ERROR:root:Caught exception: list index out of range
Saving metadata for Camera0 ...
ERROR:root:Caught exception: 'totalFrames'
Closing Camera0... Please wait.
Camera0 wrote 0 and dropped 0 frames.
SIGINT received. KeyboardInterrupt has been handled.
Closing systems...
Exiting campy...
(campy2) PS C:\Users\nalamlab\campy2>

[dankunk]

This error is fixed for me in the master branch (I am still interested in moving over to the Av1 branch but wanted to first get things working with h.265.

Essentially, I followed the tips given in #12 .

This included changing the flir.py file to comment out the white balance line, using pixel input of "gray" and output of "rgb0", adjusting the display.py file to include some additional info since we are in greyscale, and then commenting out a few more lines of code within flir.py regarding inputs.

• For the flir.py file, I made these changes:

# THIS LINE NEEDS TO BE COMMENTED OUT!
#camera.BalanceWhiteAuto.SetValue(PySpin.BalanceWhiteAuto_Off)

• I also added the function created by @chgebhardt:

def ConfigureCustomImageSettings(camera, cam_params):
	"""
	Configures a number of settings on the camera including offsets X and Y, width,
	height, and pixel format. These settings must be applied before BeginAcquisition()
	is called; otherwise, they will be read only. Also, it is important to note that
	settings are applied immediately. This means if you plan to reduce the width and
	move the x offset accordingly, you need to apply such changes in the appropriate order.
	:param nodemap: GenICam nodemap.
	:type nodemap: INodeMap
	:return: True if successful, False otherwise.
	:rtype: bool
	"""

	try:
		if cam_params["cameraDebug"]:
			print("\n*** CONFIGURING CUSTOM IMAGE SETTINGS *** \n")

		settingsConfig = True

		camera.AcquisitionMode.SetValue(PySpin.AcquisitionMode_Continuous)
		#camera.BalanceWhiteAuto.SetValue(PySpin.BalanceWhiteAuto_Off)

		cam_params = ConfigureFrameWidth(camera, cam_params)
		cam_params = ConfigureFrameHeight(camera, cam_params)
		cam_params = ConfigurePixelFormat(camera, cam_params)
		cam_params = ConfigureExposure(camera, cam_params)
		cam_params = ConfigureGain(camera, cam_params)
		cam_params = ConfigureGamma(camera, cam_params)
		cam_params = ConfigureBuffer(camera, cam_params)
		cam_params = ConfigureChunkData(camera, cam_params)
                # BE SURE TO INCLUDE THIS LINE AS WELL AS THE ConfigureFrameRate() FUNCTION BELOW!!!
		cam_params = ConfigureFrameRate(camera, cam_params)

	except Exception as e:
		logging.error("Caught error at cameras/flir.py ConfigureCustomImageSettings: {}".format(e))
		settingsConfig = False

	cam_params["settingsConfig"] = settingsConfig

	return cam_params

#HERE WE ARE ADDING THE NEW FUNCTION!
def ConfigureFrameRate(camera, cam_params):
    """Test ConfigureFrameRate function."""
    try:        
        nodemap = camera.GetNodeMap()
        # Enable change of acquisition framerate
        nodeAcquisitionFramerateEnable = PySpin.CBooleanPtr(nodemap.GetNode("AcquisitionFrameRateEnable"))
        if (not PySpin.IsAvailable(nodeAcquisitionFramerateEnable)) or (not PySpin.IsWritable(nodeAcquisitionFramerateEnable)): 
             print('Unable to retrieve AcquisitionFrameRateEnable. Aborting...')
             return -1
        nodeAcquisitionFramerateEnable.SetValue(True)
        # Set acquisition framerate
        framerate_to_set = cam_params["frameRate"]
        nodeAcquisitionFramerate = PySpin.CFloatPtr(nodemap.GetNode("AcquisitionFrameRate"))
        if not PySpin.IsAvailable(nodeAcquisitionFramerate) and not PySpin.IsWritable(nodeAcquisitionFramerate):
           print('Unable to retrieve AcquisitionFrameRate. Aborting...')
           return -1
       
        nodeAcquisitionFramerate.SetValue(framerate_to_set)
        print('Frame rate set to {}fps...'.format(framerate_to_set))
        
    except Exception as e:
        logging.error("Caught exception at cameras/flir.py ConfigureFrameRate: {}".format(e))
    
    return cam_params

• Also, comment out these lines in the flir.py file

#if cam_params["pixelFormatInput"] == "bayer_rggb8" \
		#or cam_params["pixelFormatInput"] == "bayer_bggr8" \
		#or cam_params["pixelFormatInput"] == "gray":
			# Convert to RGB
			#grabResult = grabResult.Convert(PySpin.PixelFormat_RGB8, PySpin.HQ_LINEAR)

It should now look something like this...

def DisplayImage(cam_params, dispQueue, grabResult):
	try:
		#if cam_params["pixelFormatInput"] == "bayer_rggb8" \
		#or cam_params["pixelFormatInput"] == "bayer_bggr8" \
		#or cam_params["pixelFormatInput"] == "gray":
			# Convert to RGB
			#grabResult = grabResult.Convert(PySpin.PixelFormat_RGB8, PySpin.HQ_LINEAR)

		# Convert to Numpy array
		img = GetImageArray(grabResult)

		# Downsample image
		img = img[::cam_params["displayDownsample"], ::cam_params["displayDownsample"]]

		# Send to display queue
		dispQueue.append(img)

	except Exception as e:
		logging.error('Caught exception at cameras/flir.py DisplayImage: {}.'.format(e))

• Finally, I adjusted the display.py script to include some extra info regarding the grayscale as suggested by @histun .
• It should look like this

def DrawFigure(num):
	mpl.rcParams['toolbar'] = 'None' 

	figure = plt.figure(num)
	ax = plt.axes([0,0,1,1], frameon=False)

	plt.axis('off')
	plt.autoscale(tight=True)
	plt.ion()
        #This is the updated line with cmap='gray',vmin=0,vmax=255
	imageWindow = ax.imshow(np.zeros((1,1,3), dtype='uint8'), 
		interpolation='none',cmap='gray',vmin=0,vmax=255)

	figure.canvas.draw()
	plt.show(block=False)

	return figure, imageWindow

and then here is the config that worked for me with a FLIR BFS monochrome camera. I am still some slight issues with getting configs to stay saved onto the camera but I will open a new issue for troubleshooting those issues.

# This beginner-level config uses the CPU (gpuID: -1) to compress video
# using ffmpeg's X264 codec

# Recording parameters
videoFolder: "./test"
videoFilename: "test1.mp4"
frameRate: 60
recTimeInSec: 600
chunkLengthInSec: 120

# Camera parameters
numCams: 1
cameraMake: "flir"
cameraExposureTimeInUs: 5002
frameHeight: 1152
frameWidth: 1024  

# Flir settings
cameraSettings: "UserSet1"


# Compression parameters
gpuID: 0
pixelFormatInput: "gray"
pixelFormatOutput: "rgb0"                          
gpuMake: "nvidia"                     # "nvidia", "amd", "intel"                  
codec: "h265"                         # "h264", "h265", or user-defined video codec
quality: 21                           # 0-55
ffmpegLogLevel: "quiet"               # "quiet", "warning", "info"
# ffmpegPath: "/usr/bin/ffmpeg"       # Location of ffmpeg binary if not using Anaconda


# Display parameters
displayFrameRate: 10

The true dimensions of my camera are 2048x1536 but I have been having issues with those specific frame heights and widths. I'm assuming it is an issue with camera properties getting overwritten each time I run a config file.