Tracing while generating response

[Daniel (dast-draftwise)]

Hi,
I have a specific use case involving user-initiated cancellations of agent workflows.

Context
In my frontend, users submit a query that triggers an agent run, which is traced in LangSmith.
Sometimes the user cancels the agent very early—before the LLM begins generating a response.

Problem
When the agent is canceled early, the trace_id (or run ID) has not been created yet.
Because of that, I cannot attach feedback to the trace or run indicating that the agent was canceled by the user.

Question
Is there a recommended approach or best practice for tagging / recording feedback for runs that get canceled before a trace is created?

Ideally, I’d like a way to:

• Pass a cancellation reason (e.g., “canceled_by_user”)
• Ensure it attaches to whatever run eventually appears, if any
• Or otherwise record this cancellation event in LangSmith in a meaningful way

Any guidance or patterns for handling this scenario would be appreciated.
Thanks!

[Angus Jelinek (angus-langchain)]

Daniel (@dast-draftwise) Going to convert this to a discussion since it a general question and not an actionable issue with our SDK. Will respond there

[Angus Jelinek (angus-langchain)]

When the agent is canceled early, the trace_id (or run ID) has not been created yet.

Depending on how the agent is cancelled, we should still trace the user input, you can ensure all traces are flushed by using client.flush() or wait_for_all_tracers(). You can also include an error or metadata in the traced function which can contain the cancellation reason. Then you can attach feedback to the incomplete or errored run.

[Daniel (dast-draftwise)]

Thank you for the reply.
I do not use directly from langsmith import Client in my code so I can not do client.flush() instead I am using OpenAIAgentsTracingProcessor, how would I implement proper flushing with that?

[Angus Jelinek (angus-langchain)]

Daniel (@dast-draftwise) You can pass a langsmith client into the OpenAIAgentsTracingProcessor directly, and then flush the client outside of the trace wrapper

[Daniel (dast-draftwise)]

Thank you