Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus Remote-Write receiver Alpha #37277

Open
9 tasks
dashpole opened this issue Jan 16, 2025 · 25 comments
Open
9 tasks

Prometheus Remote-Write receiver Alpha #37277

dashpole opened this issue Jan 16, 2025 · 25 comments
Assignees

Comments

@dashpole
Copy link
Contributor

Component(s)

receiver/prometheusremotewrite

Describe the issue you're reporting

  • Handle empty otel_scope_name and otel_scope_version
  • Handle Timeseries samples
  • Handle Counters
  • Handle Histograms
  • Handle Summaries (maybe too hard and could be left out for Alpha)
  • Handle Created Timestamp
  • Return number of samples and histograms ingested via response headers
  • Handle multiple timeseries with same metric name+type+unit
  • Handle target_info
@dashpole dashpole added needs triage New item requiring triage receiver/prometheusremotewrite enhancement New feature or request labels Jan 16, 2025
Copy link
Contributor

Pinging code owners for receiver/prometheusremotewrite: @dashpole @ArthurSens. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triaged' to set the priority and remove the needs-triaged label.

@adwait-godbole
Copy link

adwait-godbole commented Jan 24, 2025

Hi @ArthurSens! Hi @dashpole! I am eager to be a part of this project and looking forward to work under your mentorship and guidance. Are there any pre-requisites / pre-tests for the selection process of mentees ?

@ArthurSens
Copy link
Member

Hi Adwait, glad to see your interest. There are no pre-requisites or tests, just make sure to apply following the LFX process. Also be aware that our proposal is still just a PR, we still need both CNCF and LFX to approve the mentorship

@PiyushRajDev
Copy link

Hello @ArthurSens ! I am looking forward to contribute to this project. Can you please guide me on how to get started?

@ArthurSens
Copy link
Member

Hello @ArthurSens ! I am looking forward to contribute to this project. Can you please guide me on how to get started?

A good starting point is understanding Remote-Write and the OTLP->Prometheus compatibility spec. If you few comfortable tackling some of the checkboxes above, that's also nice :)

@hanshal101
Copy link

Hey @ArthurSens, Hanshal here. I am an open-source developer contributed to various projects like Buildsafe, Glasskube, Cyclops and a couple of Apache's too. I am currently interning at CloudRaft as a DevOps Intern. I recently published a blog on OTel Auto Instrumentation, and my curiosity towards observability brought me here. I'm looking forward to working on this issue if it gets accepted.
I am pretty much aware of the OTLP -> Prometheus spec; what else should I focus on more to understand the issue in a better way?

Thank you.

@ArthurSens
Copy link
Member

ArthurSens commented Feb 2, 2025

Hi Hanshal, this project is quite simple so we're not requiring that much experience, to be honest. There's nothing else to be familiar with besides the two protocols that were already mentioned: Prometheus Remote Write and OTLP.

We'd love to mentor someone who hasn't had a chance with OSS yet and is excited to participate in the community, that's all.

Anyone interested, just make sure you follow the LFX process correctly.

@PiyushRajDev
Copy link

Hello @ArthurSens ! I have read through the suggested documentation and familiarized myself with the protocols. As I dive deeper into the project, I may encounter some challenges along the way. Would it be possible to join a Slack channel or another communication platform where I can ask quick clarifying questions when needed? I understand you’re busy, so I really appreciate any help or pointers you can offer.

@pratikgitss
Copy link

pratikgitss commented Feb 5, 2025

I agree with @PiyushRajDev, it would be great to have a communication channel as I have also gone through the protocols and I am figuring out how to tackle the requirements mentioned above and will be really helpful if there is a channel to get the questions clarified.

@ArthurSens
Copy link
Member

Hello hello 👋 -- This issue is a great communication channel! Feel free to ask questions here :)

But yes, there is also the CNCF slack. We have a channel called #otel-prometheus that can also be used to ask questions. Asking it here would be a lot better though, since the information would be accessible to all.

@sonalgaud12
Copy link

Hey everyone, I'm Sonal! Recently, I've been contributing to the OTEL Collector and OTEL Collector Contrib, specifically under the Receiver package. I'm really excited about this project and believe there's so much to learn from it. By the way, are there any upcoming meetings or briefings about the project? I'd love to contribute more!

@PiyushRajDev
Copy link

Hello @sonalgaud, there are not any meeting or briefings as such but we need to just apply through the LFX process for this issue.

@Brant-lzh
Copy link

Hello, I just need this feature and I am very eager to participate in this project, so what do I need to do now?

@PiyushRajDev
Copy link

Hey @Brant-lzh, for getting started you need to refer to one of the messages sent by @ArthurSens up above on this thread where he sent links of the two protcols for getting started.

@Brant-lzh
Copy link

Hey @PiyushRajDev ,I understand these contents. I have a certain understanding of prometheus and opentelemetry. How should I start now? Is there a working group?

@sonalgaud12
Copy link

Hey @Brant-lzh , this project will be developed under the LFX mentorship program. You’ll need to apply for it, and the selected candidate will be working on it. For communication, you can join the CNCF Slack channel named otel-prometheus.

@Brant-lzh
Copy link

Brant-lzh commented Feb 8, 2025

Hey @sonalgaud12 , I joined the Slack Channel, and how to apply for this LFX mentorship program? Is there any relevant documentation?

@sonalgaud12
Copy link

sonalgaud12 commented Feb 8, 2025

@Brant-lzh
Copy link

@sonalgaud12 thx!

@PiyushRajDev
Copy link

Hey @ArthurSens , Could you please point me to any existing implementations or helper functions in the code that follow a similar structure to addCounterDatapoints? It would be helpful to review how similar cases are handled elsewhere in the code for consistency.

@ArthurSens
Copy link
Member

It should be very very similar to addGaugeDatapoints, but I wouldn't implement this first though, let's first handle the scope attributes or the samples.

@sonalgaud12
Copy link

sonalgaud12 commented Feb 11, 2025

Hey @ArthurSens , for the first to do I could think of two possible solution.

  1. Use Collector Build Tags
  2. Use a Default Fallback Value (If Build Tags Are Not Available)
    As the comment suggest the collector build approach do let me know which one will be better I'll raise an PR.

@PiyushRajDev
Copy link

Hi @sonalgaud12 , There's already a PR addressing this: #37791

The PR is currently implementing the Collector Build Tags approach using settings.BuildInfo.
Perhaps we could collaborate there and help review the existing implementation?

@sonalgaud12
Copy link

I see, actually I was thinking of adding an scopedetail func to fetch data from collector settings

@PiyushRajDev
Copy link

That's an interesting approach, having a scopedetail function could make the code more maintainable and could be reused in both addGaugeDatapoints and addCounterDatapoints.
It could work well with the current PR's approach of using settings.BuildInfo.
Would you like to suggest this in the PR discussion?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants