test: API tests intermittently fail when executing from Apple Silicon

[jamestexas]

Description

There are a few tests that fail on the latest release (0.11.0 of leapfrogai_api) that are in the test_api.py file. These can be broken up into 2 logical groups:

• Config deletion behaving flakily. This is due to a race condition that spawns from the Config class using watchfiles. Watchfiles uses anyio as the backend for it's async operations, and specifically uses a thread to manage it's "file watch" task. The config deletion step of the test has a sleep present to allow the "update" to be processed by the API. This does not work on a Mac with Apple Silicon, but does appear to work in Linux.

• Requests with auth not working properly. This is due to a few things. There are 2 problems that are causing this:

  • The global app state is being modified / relied on to execute this test. Because we define app as a global in main.py this means that these modifications could have side effects if other tests / this environment is sustained.
  • The localhost definition in the test config file appears to be causing failures when making GRPC requests. This appears to be solved by switching the host to 0.0.0.0 in the config file.

Environment

1. OS and Architecture:

└> sw_vers; uname -m
ProductName:		macOS
ProductVersion:		14.6.1
BuildVersion:		23G93
Architecture:           arm64

2. Package Name and Version:

• Name: leapfrogai_api
• Version: 0.11.0

3. Kubernetes Distribution: K3D
4. Kubernetes Version:

└> kubectl version --client
Client Version: v1.30.2
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3

5. Other:

Steps to reproduce

1. With a locally configured dev virtual environment, run LFAI_RUN_REPEATER_TESTS=true python3 -m pytest -vv 'tests/pytest'

Expected result

• All tests pass

Actual Result

FAILED tests/pytest/leapfrogai_api/test_api.py::test_config_delete - AssertionError: assert {'config_sour... 'repeater'}}} == {'config_sour... 'models': {}}
FAILED tests/pytest/leapfrogai_api/test_api.py::test_embedding - assert 405 == 200
FAILED tests/pytest/leapfrogai_api/test_api.py::test_chat_completion - assert 405 == 200
FAILED tests/pytest/leapfrogai_api/test_api.py::test_stream_chat_completion - assert 405 == 200

Visual Proof (screenshots, videos, text, etc)

Additional Context

Add any other context or screenshots about the technical debt here.