Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HRNet stopped working after NumPy 1.24 #5571

Closed
2 tasks done
rodrigoberriel opened this issue Jan 11, 2023 · 2 comments · Fixed by #5574
Closed
2 tasks done

HRNet stopped working after NumPy 1.24 #5571

rodrigoberriel opened this issue Jan 11, 2023 · 2 comments · Fixed by #5574

Comments

@rodrigoberriel
Copy link
Contributor

rodrigoberriel commented Jan 11, 2023

My actions before raising this issue

Three weeks ago NumPy 1.24.0 was released and they finally expired the deprecation for np.int introduced in Numpy 1.20:

The deprecation for the aliases np.object, np.bool, np.float,
np.complex, np.str, and np.int is expired (introduces NumPy
1.20). Some of these will now give a FutureWarning in addition to
raising an error since they will be mapped to the NumPy scalars in
the future.

Therefore, using np.int now results in an error instead of a warning. It just happens that HRNet uses np.int and the NumPy version is not pinned in their requirements.txt. So, if you try to deploy HRNet today, it will pull NumPy >= 1.24 and the following error will be thrown:

AttributeError: module 'numpy' has no attribute 'int'

23.01.11 15:42:04.675 sor.http.w0.python.logger (E) Exception raised while running init_context {"worker_id": "0"}
Traceback (most recent call last):
  File "/opt/nuclio/_nuclio_wrapper.py", line 432, in <module>
    run_wrapper()
  File "/opt/nuclio/_nuclio_wrapper.py", line 420, in run_wrapper
    loop.run_until_complete(wrapper_instance.initialize())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/opt/nuclio/_nuclio_wrapper.py", line 147, in initialize
    await self._initialize_context()
  File "/opt/nuclio/_nuclio_wrapper.py", line 158, in _initialize_context
    init_context_result = getattr(self._entrypoint_module, 'init_context')(self._context)
  File "/opt/nuclio/main.py", line 14, in init_context
    model = ModelHandler()
  File "/opt/nuclio/model_handler.py", line 45, in __init__
    self.net = utils.load_is_model(checkpoint_path, self.device)
  File "/opt/nuclio/hrnet/isegm/inference/utils.py", line 33, in load_is_model
    return load_single_is_model(state_dict, device, **kwargs)
  File "/opt/nuclio/hrnet/isegm/inference/utils.py", line 37, in load_single_is_model
    model = load_model(state_dict['config'], **kwargs)
  File "/opt/nuclio/hrnet/isegm/utils/serialization.py", line 64, in load_model
    return model_class(**model_args)
  File "/opt/nuclio/hrnet/isegm/utils/serialization.py", line 39, in new_init
    init(self, *args, **kwargs)
  File "/opt/nuclio/hrnet/isegm/model/is_hrnet_model.py", line 15, in __init__
    self.feature_extractor = HighResolutionNet(width=width, ocr_width=ocr_width, small=small,
  File "/opt/nuclio/hrnet/isegm/model/modeling/hrnet_ocr.py", line 208, in __init__
    last_inp_channels = np.int(np.sum(pre_stage_channels))
  File "/usr/local/lib/python3.8/dist-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'int'
23.01.11 15:42:04.928 sor.http.w0.python.logger (W) Failed to read from connection {"err": "EOF"}
23.01.11 15:42:04.931 sor.http.w0.python.logger (E) Unexpected termination of child process {"error": null, "status": "exit status 1"}

Expected Behaviour

HRNet should be deployed with success.

Current Behaviour

The HRNet does not deploy successfully as of 3 weeks ago.

Possible Solution

Add an extra step to function-gpu.yaml, downgrading NumPy: pip install "numpy<1.24".

I tested it in my environment and it worked. I can submit a PR if this approach is okay.

Alternative:

  • Patch the requirements.txt before installing it, replacing the numpy line to numpy<1.24.

Steps to Reproduce (for bugs)

  1. Deploy HRNet
  2. Try to use it
  3. Check the logs: docker logs nuclio-nuclio-pth-saic-vul-hrnet

Context

I re-deployed HRNet after #5570, and it didn't work.

Your Environment

  • Git hash commit (git log -1): 43ed0a2
  • Docker version docker version (e.g. Docker 17.0.05): 20.10.18
  • Are you using Docker Swarm or Kubernetes? No.
  • Operating System and version (e.g. Linux, Windows, MacOS): Linux.
@nmanovic
Copy link
Contributor

@rodrigoberriel , thanks for the report. Could you please send us a PR?

@rodrigoberriel
Copy link
Contributor Author

@rodrigoberriel , thanks for the report. Could you please send us a PR?

Done!

nmanovic pushed a commit that referenced this issue Mar 23, 2023
HRNet uses `np.int` which throws an error for NumPy >= 1.24, because its
deprecation was finally expired. This PR creates an additional step
during HRNet function image build, downgrading NumPy for a version <
1.24.

### Motivation and context
Closes #5571.
mikhail-treskin pushed a commit to retailnext/cvat that referenced this issue Jul 1, 2023
HRNet uses `np.int` which throws an error for NumPy >= 1.24, because its
deprecation was finally expired. This PR creates an additional step
during HRNet function image build, downgrading NumPy for a version <
1.24.

### Motivation and context
Closes cvat-ai#5571.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants