How to pass, in python, a parameter not provided by the LLM to a function?

[ja-bot]

Hello,

We are developing an agent that, based on a user request, queries several APIs for getting information that answers the user request. These API calls require a user access key (that is unique per user) and that we receive with the user request. We don't want the LLM to reproduce that user key for filling the parameters of the functions that it is calling. Instead, we would like the LLM to fill parameters related to the user request, but we want to fill the API key programmatically, to avoid any issue with this parameter.

If I am not wrong, this should be done through "kernel function transformations", am I correct? If yes, I only found an example doing this in C# ( https://devblogs.microsoft.com/semantic-kernel/transforming-semantic-kernel-functions/ ), is this functionality available in Python?

Thanks!

[moonbox3]

Hi @ja-bot, thanks for your question. Right now we don't have the direct 1:1 "kernel function transformation" method as you pointed out that C# has. Instead, you can emulate this in Python by registering a custom invocation filter that intercepts arguments, removes or masks the sensitive key, and then passes control to the next filter or the underlying function. This way, the LLM never sees that key.

I haven't explicitly tested this yet, but just trying to think through some code that may work:

async def remove_api_key_filter(context: FunctionInvocationContext, next_filter) -> None:
    """A filter that intercepts the function invocation context and
    removes the 'api_key' argument so that it's not passed to the LLM."""

    if 'api_key' in context.arguments:
        logger.info("Removing api_key before sending to LLM.")
        # save a copy for later if you do want to store it on the context
        # or your own state (just not in the user-facing arguments)
        user_api_key = context.arguments['api_key']
        
        # remove from the user arguments so the LLM can’t see or “accidentally” get it
        del context.arguments['api_key']

    # call the next filter or actual function in the chain
    await next_filter(context)

One thought is: do you remove and store the key for later use? Something like:

async def remove_api_key_filter(context: FunctionInvocationContext, next_filter) -> None:
     """A filter that intercepts the function invocation context and
    removes the 'api_key' argument so that it's not passed to the LLM."""
    
    if "api_key" in context.arguments:
        user_api_key = context.arguments["api_key"]
        del context.arguments["api_key"]

        if context.result is None:
            # If we haven't created one yet, do so
            from semantic_kernel.functions.function_result import FunctionResult
            context.result = FunctionResult(function=context.function.metadata, value=None)

        # Attach the key to the result's metadata
        context.result.metadata["api_key"] = user_api_key

    await next_filter(context)

You'd register the filter on the kernel like:

kernel.register_filter(
    filter_type=FilterTypes.FUNCTION_INVOCATION,
    filter=remove_api_key_filter,
)

I'm kind of basing these thoughts on the following sample: https://github.com/microsoft/semantic-kernel/blob/main/python/samples/concepts/filtering/function_invocation_filters.py

Again, I'd need to play around with it for a bit, but maybe this can get you on the right path. Let me know if you have any questions about this.

[ja-bot]

Thanks @moonbox3! Let us test this!