[Python][DSPy] trace LM Calls

[mikeldking]

Add instrumentation for the language model calls

[dosubot]

🤖

Sure, I can guide you on how to add instrumentation for the language model calls in Python using DSPy in the OpenInference repository.

Firstly, you need to import the necessary modules. This includes openai for the language model, OpenAIInstrumentor from openinference.instrumentation.openai for the instrumentation, and various modules from opentelemetry for the tracing.

Next, you need to set up the tracing provider. This involves creating a Resource object, a TracerProvider object, an OTLPSpanExporter object, and a SimpleSpanProcessor object. The OTLPSpanExporter is set up to send traces to the endpoint at http://127.0.0.1:6006/v1/traces. The SimpleSpanProcessor is set up with the OTLPSpanExporter, and is then added to the TracerProvider. Finally, the TracerProvider is set as the global tracer provider.

After setting up the tracing provider, you need to instrument the OpenAI library. This is done by creating an OpenAIInstrumentor object and calling its instrument method.

Finally, you can make the language model call. This is done by creating an OpenAI object and calling its embeddings.create method with the desired model and input. The response is then printed.

Here is a code snippet that demonstrates these steps:

import openai
from openinference.instrumentation.openai import OpenAIInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

resource = Resource(attributes={})
tracer_provider = trace_sdk.TracerProvider(resource=resource)
span_exporter = OTLPSpanExporter(endpoint="http://127.0.0.1:6006/v1/traces")
span_processor = SimpleSpanProcessor(span_exporter=span_exporter)
tracer_provider.add_span_processor(span_processor=span_processor)
trace_api.set_tracer_provider(tracer_provider=tracer_provider)

OpenAIInstrumentor().instrument()

if __name__ == "__main__":
    response = openai.OpenAI().embeddings.create(
        model="text-embedding-ada-002",
        input="hello world",
    )
    print(response.data[0].embedding)

This code will instrument the OpenAI library and trace the language model call, sending the trace data to the specified endpoint. You can find more details in the embeddings.py example in the OpenInference repository.

As for the OpenAIInstrumentor class, it is an instrumentor for OpenAI that extends the BaseInstrumentor class from the opentelemetry.instrumentation.instrumentor module. Its main role is to instrument the OpenAI requests for tracing purposes by wrapping the OpenAI.request and AsyncOpenAI.request methods with the _Request and _AsyncRequest wrappers respectively. These wrappers are used to add tracing functionality to the requests. You can find more details in the OpenAIInstrumentor source code in the OpenInference repository.

I hope this helps! If you have any further questions, feel free to ask.

Sources

• python/instrumentation/openinference-instrumentation-openai/examples/embeddings.py
• python/instrumentation/openinference-instrumentation-openai/src/openinference/instrumentation/openai/init.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.