Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Archiver queried for values even if ncount=0 #2575

Open
georgweiss opened this issue Mar 10, 2023 · 3 comments
Open

Archiver queried for values even if ncount=0 #2575

georgweiss opened this issue Mar 10, 2023 · 3 comments

Comments

@georgweiss
Copy link
Collaborator

In ApplianceArchiverReader L140 the archiver(s) are queried for the number of data points calling getNumberOfPoints which - no points are found - will call getNumberOfPointsLegacy.

Now if this chain of calls returns zero, then the archiver is called to retrieve raw values anyway.

So I have some questions:

  1. When is getNumberOfPointsLegacy needed? Is this related to certain versions of the archiver appliance? If so, then it should be possible to omit the call if only newer versions are used.
  2. If the count is zero, should not the call for values be omitted completely?
  3. Since retrieving values will contain information on the number of values, why call getNumberOfPoints in the first place?

I'm asking this since we are migrating to a new clusters, but we're not migrating data. So Phoebus needs to query twice as many endpoints for each plot, putting load on both the application and the archiver services. And network.

@kasemir
Copy link
Collaborator

kasemir commented Mar 10, 2023

When is getNumberOfPointsLegacy needed?

No idea, that's internal to the archive appliance client implementation

If the count is zero, should not the call for values be omitted completely?

Yes, if you know that there are no values, then the call could return an empty value iterator

Since retrieving values will contain information on the number of values, why call getNumberOfPoints in the first place?

The goal here is to decide if we can return the raw data (count is below some threshold "count"), or if we should return "optimized" min/max/average type of data because there are way too many rows of raw data.

@georgweiss
Copy link
Collaborator Author

Sure, but ncount=0 should indicate that there's no point getting raw data at all. Right?

I'm asking this since we have added new archiver clusters and now have a total of 4. Which means each plot will make the data browser send off a lot of requests. In view of how we organize these clusters over time only one of them will have the wanted data.

@kasemir
Copy link
Collaborator

kasemir commented Mar 16, 2023

Yes, if you know that there are no values, then the call could return an empty value iterator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants