OpenRouter support for Langsmith

[kvyb]

Feature request

OpenRouter is very popular as a provider for LLM inference, and should be available for use in playground with all of its models, next to OpenAI etc.

Motivation

Its a pity that it is not available in LangSmith Playground. Meaning I can't really do experiments with traces since all of our production inference is done on OpenRouter LLMs, and some of these LLMs are not even available with other providers.

[William FH (hinthornw)]

Will add to backlog, and if this ticket gets a lot of demand we can re-prioritize

[kvyb]

Thanks

[Mickey (hahaCrazy)]

Some providers of openrouter are very unstable, and problems often occur in the model. If you do not specify a provider, when calling the api, the providers will be sorted by openrouter's default order. This order changes dynamically. Therefore, it is often easy for the model to fail to work properly.

The order in which langchain adds providers to openrouter is really important!

[Arvin (arvin8)]

Measuring feature demend based on github tickets can't be seen as a good product practice. There should be demand since openrouter is a very popular service and is the only service. But the demend for it might not be visible on this ticket which is open on a development platform like github. Demend here will come from developers but choice of tool will rarely come from the coding developer but higher managment.

[Wing Lian (winglian)]

Is it not sufficient to simply change the api base to point to openrouter for langsmith integration to work?

[William FH (hinthornw)]

If it is openai compatible than it would be

[kvyb]

It seems to be in fact openai compatible

[Mickey (hahaCrazy)]

Some providers of openrouter are very unstable, and problems often occur in the model. If you do not specify a provider, when calling the api, the providers will be sorted by openrouter's default order. This order changes dynamically. Therefore, it is often easy for the model to fail to work properly.

The order in which langchain adds providers to openrouter is really important!

[Arvin (arvin8)]

need to have an option to define the provider as well.

[Jorben (jorben)]

Is there any news about this feature?

[Facu Gamond (facu-gamond)]

This works well if you set the base URL and paste your OpenRouter API key. However, if you add the model directly to the registry for pulling, it doesn’t work — so I had to create an additional service to call the LLM manually. The downside is that you lose cost tracking in the tracing panel, which I guess is expected since I’m using a model from a different SDK provider.

Here's what I had to do to make it works:

// Pull prompt and model as a chain
    const chain = await hub.pull<Runnable>(promptName, { includeModel: true });

    // Force OpenRouter configuration if using ChatOpenAI
    let openRouterModel: any = null;
    if (chain.constructor.name === "RunnableSequence") {
      const sequence = chain as any;
      if (sequence.last?.bound?.constructor.name === "ChatOpenAI") {
        const originalModel = sequence.last.bound;

        // Create new ChatOpenAI instance with correct OpenRouter configuration
        openRouterModel = new ChatOpenAI({
          modelName: originalModel.model,
          temperature: originalModel.temperature,
          maxTokens: originalModel.maxTokens,
          topP: originalModel.topP,
          presencePenalty: originalModel.presencePenalty,
          frequencyPenalty: originalModel.frequencyPenalty,
          openAIApiKey: process.env.OPENROUTER_API_KEY,
          configuration: {
            baseURL: "https://openrouter.ai/api/v1",
            defaultHeaders: {
              "HTTP-Referer": "http://localhost:3000",
              "X-Title": "Buddd",
            },
          },
        });

        // Add JSON mode if requested
        if (jsonMode) {
          openRouterModel = openRouterModel.bind({
            response_format: {
              type: "json_object",
            },
          });
        }

        if (tools && tools.length > 0) {
          openRouterModel = openRouterModel.bindTools(tools);
        } else {
          // Replace the model in the chain
          sequence.last.bound = openRouterModel;
        }
      }
    }

    // If we have tools, use the model with tools instead of the chain
    if (tools) {
      // Get the prompt from the chain and invoke with the tools-enabled model
      const sequence = chain as any;
      const prompt = sequence.first; // The prompt is usually the first element

      const result = await prompt.pipe(openRouterModel).invoke(variables);

      //console.log(`📤 LangSmith response with tools:`, result);
      return result; // Return the full AIMessage with potential tool_calls
    }

    // Fallback to original chain without tools
    const result = await chain.invoke(variables);

    // Use custom parser if provided
    if (customParser) {
      return await customParser.parse(result.content);
    }

    // Use built-in JSON parsing for jsonMode
    if (jsonMode) {
      try {
        return JSON.parse(result.content);
      } catch (error) {
        // Try to extract JSON from markdown code blocks as fallback
        const jsonMatches = result.content.match(
          /```(?:json)?\s*([\s\S]*?)\s*```/
        );
        if (jsonMatches) {
          return JSON.parse(jsonMatches[1].trim());
        }
      }
    }

    return result.content;

[Angus Jelinek (angus-langchain)]

Adding this as OpenRouter announced Broadcast recently https://openrouter.ai/docs/guides/features/broadcast/langsmith