Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adapt download_acoustic_dataset() for use by OpenCPU #32

Closed
5 tasks done
PietrH opened this issue Jun 12, 2023 · 3 comments
Closed
5 tasks done

Adapt download_acoustic_dataset() for use by OpenCPU #32

PietrH opened this issue Jun 12, 2023 · 3 comments
Assignees
Labels
dev Improvements to development workflow enhancement New feature or request

Comments

@PietrH
Copy link
Member

PietrH commented Jun 12, 2023

download_acoustic_dataset() is one of only two functions within etn that doesn't (exclusively) return an object, the other being write_dwc()

This issue is meant to track the decision process and progress in adapting this function for use by OpenCPU, specifically the division of subtasks between the client and the API service.

In short:

download_acoustic_dataset() creates 8 objects:

  1. if it doesn't exist, a new directory to write to
  2. animals object
  3. tags object (+ tag_serial_numbers)
  4. detections object
  5. deployment object (+ acoustic_project_codes)
  6. receivers object (+ receiver_ids)
  7. datapackage.json which is static and can be included in the client package
  8. messaging: summary statistics, warning and progress messaging

of these, animals, tags, detections, deployments, and receivers get written out as csv. datapackage.json is copied over, it makes sense to just include this in the client.

Now, animals, tags, detections, deployments, and receivers can either all be generated by the API, and returned as a single list object via RDA. Or the client can make them itself via calls to get_animals(), get_tags(), get_acoustic_detections(), get_acoustic_deployments() and get_acoustic_receivers()

passing messaging over the API is possible by retreiving STDOUT, but some formatting is likely to get lost, and it'll get returned as a single blob of text. So it will not properly report progress. From this point of view it makes more sense to have the client make multiple requests and report on progress in between.

Options:

A. Make a lot of requests, maintain messaging and progress reporting
B. Make single request, but possibly tie up API for a while, probably lose progress reporting

Dependencies:

The implementation of this function is dependent on the previous implementation of the following functions:

  • get_animals()
  • get_tags()
  • get_acoustic_detections()
  • get_acoustic_deployments()
  • get_acoustic_receivers()

And their respective dependencies.

@PietrH PietrH added enhancement New feature or request dev Improvements to development workflow labels Jun 12, 2023
@PietrH PietrH self-assigned this Jun 12, 2023
@PietrH
Copy link
Member Author

PietrH commented Jun 12, 2023

general progress is tracked in inbo/etn#281

@PietrH
Copy link
Member Author

PietrH commented Jun 13, 2023

I've slept over it, and decided I want to initially develop this function client side. To preserve progress reporting, and to free up the API as quickly as possible (more small calls rather than single bigger ones). But either option could be transitioned into the other with some effort.

@PietrH
Copy link
Member Author

PietrH commented Jun 19, 2023

Implemented on the client side

@PietrH PietrH closed this as completed Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev Improvements to development workflow enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant