Cannot train Recognizer - ValueError: Exception encountered when calling layer 'lambda_1' (type Lambda).

[ACRae]

Trying to run the https://colab.research.google.com/drive/19dGKong-LraUG3wYlJuPCquemJ13NN8R#scrollTo=27hytnFasGBO
I cannot seem to train the recognizer - recognizer.training_model.fit_generator.
Code:

recognition_batch_size = 8
recognizer_basepath = os.path.join(data_dir, f'recognizer_{datetime.datetime.now().isoformat()}')
recognition_train_generator, recognition_val_generator, recogntion_test_generator = [
    recognizer.get_batch_generator(
      image_generator=image_generator,
      batch_size=recognition_batch_size,
      lowercase=True
    ) for image_generator in recognition_image_generators
]
recognizer.training_model.fit_generator(
    generator=recognition_train_generator,
    epochs=1000,
    steps_per_epoch=math.ceil(len(background_splits[0]) / recognition_batch_size),
    callbacks=[
      tf.keras.callbacks.EarlyStopping(restore_best_weights=True, patience=25),
      tf.keras.callbacks.CSVLogger(f'{recognizer_basepath}.csv', append=True),
      tf.keras.callbacks.ModelCheckpoint(filepath=f'{recognizer_basepath}.h5')
    ],
    validation_data=recognition_val_generator,
    validation_steps=math.ceil(len(background_splits[1]) / recognition_batch_size),
    workers=0
)

Output:

<ipython-input-14-f7617c42bda1>:10: UserWarning: `Model.fit_generator` is deprecated and will be removed in a future version. Please use `Model.fit`, which supports generators.
  recognizer.training_model.fit_generator(
Epoch 1/1000
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-14-f7617c42bda1>](https://localhost:8080/#) in <cell line: 10>()
      8     ) for image_generator in recognition_image_generators
      9 ]
---> 10 recognizer.training_model.fit_generator(
     11     generator=recognition_train_generator,
     12     epochs=1000,

2 frames
[/usr/local/lib/python3.9/dist-packages/keras/engine/training.py](https://localhost:8080/#) in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---> 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1284, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1268, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1249, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.9/dist-packages/keras/engine/training.py", line 1050, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.9/dist-packages/keras/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None

    ValueError: Exception encountered when calling layer 'lambda_1' (type Lambda).
    
    None values not supported.
    
    Call arguments received by layer 'lambda_1' (type Lambda):
      • inputs=['tf.Tensor(shape=(None, None, None, 512), dtype=float32)', 'tf.Tensor(shape=(None, 6), dtype=float32)']
      • mask=None
      • training=True

[busedagidir]

any solutions?

[odaon]

I am experiencing the same issue.
Any tips on how to fix this?

[odaon]

I got it workingv (not crashing) with the following changes:

import tensorflow as tf    
tf.compat.v1.disable_eager_execution()
tf.compat.v1.experimental.output_all_intermediates(True)
 

and removing batch_size from fit()

[ACRae]

I got it workingv (not crashing) with the following changes:

import tensorflow as tf    
tf.compat.v1.disable_eager_execution()
tf.compat.v1.experimental.output_all_intermediates(True)
 

and removing batch_size from fit()

Seems to have fixed it.
Got an explanation for the fix?
Also got this warning now:

WARNING:tensorflow:Discrepancy between trainable weights and collected trainable weights, did you set `model.trainable` without calling `model.compile` after ?

[geo-tp]

I had exactly the same problem. I forked this repo to make it usable with python3.10 and it also fix this

See: https://github.com/geo-tp/image-ocr
Colab training example: Training Recognizer