Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parameter Contexts blocking export-import of versioned flows #229

Closed
Chaffelson opened this issue Nov 7, 2020 · 2 comments
Closed

Parameter Contexts blocking export-import of versioned flows #229

Chaffelson opened this issue Nov 7, 2020 · 2 comments
Assignees

Comments

@Chaffelson
Copy link
Owner

From NiFi Mailing list:

HI Team,

We are facing the below mentioned issue while deploying the Nifi flow from one Nifi Registry bucket to another bucket using Nipy API, if flow contains any parameter context. However, it is working fine if flow doesn’t contain any parameter context.

Issue is while calling nipyapi.versioning.deploy_flow_version API with following error message

raise ApiException(http_resp=r)

nipyapi.nifi.rest.ApiException: (500)

Reason: Internal Server Error

HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 20 Oct 2020 07:55:30 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'Content-Security-Policy': "frame-ancestors 'self'", 'X-XSS-Protection': '1; mode=block', 'Strict-Transport-Security': 'max-age=31540000', 'Content-Type': 'text/plain', 'Vary': 'Accept-Encoding', 'Content-Length': '79', 'Server': 'Jetty(9.4.19.v20190610)'})

HTTP response body: An unexpected error has occurred. Please check the logs for additional details.

There is no error/exception in any other additional logs.

It seems that there is some issue in exporting and importing the flow from source bucket to destination bucket using Nipy API. As after importing the flow in destination bucket, it is also giving error while importing it at destination Nifi instance using Nifi UI from destination bucket.

And it is working fine while importing the flow through destination Nifi UI from source bucket. But we are not getting any error in nipyapi.versioning.export_flow_version and nipyapi.versioning.import_flow_version API calls

Could you please look into it and suggest if it is not possible through Nipy API if flow contains any parameter context. Or is there any other way to do it.

I have attached the source code for the reference.
We are using following versions:
Docker Nifi – 1.11.4
Docker Nifi Registry – nifi-registry-0.7.0
Nipy API - 0.14.3

In order to properly test this we should add Parameter Context handling to NiPy, construct tests for it, then add tests to the versioning code for this case. I'll make a start on the Parameter contexts today

@Chaffelson
Copy link
Owner Author

Controls for Parameter Contexts added in Next branch, https://github.com/Chaffelson/nipyapi/blob/next/nipyapi/parameters.py

Will look into test case for this tomorrow

@Chaffelson
Copy link
Owner Author

Have started investigating, looks like I need to make a test harness for situations involving two separate NiFi/Registry environments so we can test import/export between them. This would also help us test for nested Process Groups and a few a other usecases that people have asked about.

@Chaffelson Chaffelson self-assigned this Nov 12, 2020
Chaffelson added a commit that referenced this issue Dec 10, 2020
Found that importing a versioned flow was stripping out fields added in newer versions of NiFi.
Also found that older versions of registry do not support these Fields and were also being silently stripped, implemented version checks and warnings for old registry or NiFi not supporting Parameters.
Chaffelson added a commit that referenced this issue Dec 11, 2020
* fixes #229
Found that importing a versioned flow was stripping out fields added in newer versions of NiFi.
Also found that older versions of registry do not support these Fields and were also being silently stripped, implemented version checks and warnings for old registry or NiFi not supporting Parameters.

* Bump version: 0.16.0 → 0.16.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant