Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update NiPyAPI to be roughly forward compatible with NiFi-2.x line. #368

Merged
merged 3 commits into from
Oct 8, 2024

Conversation

Chaffelson
Copy link
Owner

Updates NiPyAPI to be roughly forward compatible with NiFi-2.x line.
Note that this does not implement the full API Clients for NiFi-2.x and NiFi-Registry-2.x, it only ensures that calls made using this client generated from the 1.27.0 swagger definition appear to work correctly according to existing tests.

  • Instruct pytest to not report insecure http requests to localhost for docker testing
  • Templates are deprecated in nifi-2x, remove from tests for newer versions
  • Add function to check registry version using newer aboutapi() call
  • Update utils.check_version to ignore any non-semver elements of a version string and use newer registry version api call if available
  • Update registry client creation to handle nifi-2x structure
  • Update Testing to treat NiFI-2.x as latest
  • Deprecate very old nifi-1x versions from regression testing
  • Fix get processor test to use a processor available in all nifi versions, GenerateFlowfile

instruct pytest to not report insecure http requests to localhost for docker testing
Templates are deprecated in nifi-2x, remove from tests for newer versions
add function to check registry version using newer aboutapi() call
update utils.check_version to ignore any non-semver elements of a version string and use newer registry version api call if available
Update registry client creation to handle nifi-2x structure
Update Testing to treat NiFI-2.x as latest
Deprecate very old nifi-1x versions from regression testing
Fix get processor test to use a processor available in all nifi versions, GenerateFlowfile
Copy link
Collaborator

@ottobackwards ottobackwards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of high level thoughts here. According to nifi, the 1.x line and the 2.x line are concurrent release paths. I'm not sure just having a "latest" and having it be 2.x-M is right here, as I don't know when the end of 1.x will be.

Also, It isn't clear to me, and it isn't in the readme, what the clear status of 2x is, or rather what the answer is to "Is this the official 2.x support release?".

I think the disposition wrt 2.x and the expectations or limitations need to be stated.
thoughts?

@Chaffelson
Copy link
Owner Author

Chaffelson commented Oct 7, 2024

A couple of high level thoughts here. According to nifi, the 1.x line and the 2.x line are concurrent release paths. I'm not sure just having a "latest" and having it be 2.x-M is right here, as I don't know when the end of 1.x will be.

yeah good point. Latest was just a convenience inherited from typical docker notation.
Perhaps it would be best to keep the 1.x line as the 'latest' for now, but put in the 2.0.0-Mx line as tested alternates? That's an easy modification.

Also, It isn't clear to me, and it isn't in the readme, what the clear status of 2x is, or rather what the answer is to "Is this the official 2.x support release?".

NiFi-2.x has moved to OpenAPI-3.0.1 as the API definition standard, which is going to require a major rewrite.
I think probably what needs to happen is that split into a 1.x maint branch that keeps the Swagger-2.0 base clients derived from the NiFi-1.x line going, and start up a 2.x branch for moving to OpenAPI3.

Thus far I've managed to keep both backwards Py2 compatibility and forwards NiFi-2.x compatibility, but it's getting very difficult these days.

I suspect we'll be moving to a 1.x release of NiPyAPI which is python3 only along with moving to OpenAPI 3.0.1 based clients from NiFi-2.x as it seems an obvious point to cut over.

So for this 0.21.0 release, we do not officially support NiFi-2.x, but it seems to work and tickets opened for support will be investigated - does that sound good?

Switch 'latest' dockerfile to use 'latest' nifi tags.
Use 2.x line in full regression testing.
@Chaffelson
Copy link
Owner Author

@ottobackwards updated the readme and docker test files as noted.

@Chaffelson Chaffelson merged commit 3010452 into main Oct 8, 2024
2 checks passed
@Chaffelson Chaffelson deleted the nifi2xCompat branch October 13, 2024 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants