[Feature] Human-in-the-loop approval as a first-class, checkpointed Action primitive #1067

Activity

1. wenjin272 commented on Aug 29, 2026

   

   wenjin272

   on Aug 29, 2026

   Contributor

   More actions

   Thanks for raising this. I agree that durable human approval could be a strong use case for Flink Agents. However, this affects the public Action contract, runtime scheduling, external signal routing, and checkpoint/recovery semantics, so I think we should agree on a concrete design in a Discussion before starting implementation.

   In particular, the current runtime introduces several questions that an await_approval API alone would not answer:

   • An unfinished ActionTask keeps the key in flight, and later events for the same key are buffered. An approval delivered as an ordinary keyed input event could therefore be queued behind the Action it needs to resume.
   • A long-lived pending key may affect watermark progression and end-of-input handling.
   • Checkpoints persist operator-managed task state, but not an in-memory Java stack or Python coroutine. Recovery therefore needs an explicit persisted approval state machine and deterministic re-entry, rather than assuming execution resumes at the original call site.
   • [Tech Debt] Align Java and Python recovery model #1016 aligns part of the Java/Python recovery model, but does not yet define durable Action suspension and external resumption.

   Since this would introduce significant changes to the Action API and runtime semantics, I think we should discuss and agree on the design before moving toward implementation. Could you please open a GitHub Discussion with a concrete design proposal so the community can review it and align on the approach?
2. dangzitou commented on Oct 21, 2026

   

   dangzitou

   on Oct 21, 2026

   More actions

   Now that DecisionModel resources have landed (#1243 / #1264), I'd like to add one angle for the eventual Discussion: treat the approver as pluggable, with a decision model as the first tier and a human as the fallback.

   Use case. In cross-border e-commerce, for example, an agent proposes actions such as raising an ad bid or pausing a listing. Most proposals are routine and could be cleared automatically; only ambiguous or high-impact ones need a person. Sending every proposal to a human makes approval the bottleneck, and every one of them would take the hard suspend/resume path described above.

   What already works today, with no new runtime semantics:

   1. The proposing Action emits a DecisionRequestEvent with a PredicateQuestion (e.g. "Is this change within policy?") and application-supplied criteria.
   2. The DecisionResponseEvent handler applies application-owned thresholds:
      • probability_true >= T_approve → continue to the next step (e.g. emit the event that triggers the tool call)
      • probability_true <= T_reject → reject and emit an output
      • in between, a RefusalAnswer, or a failed response → escalate to a human

   Because the call goes through the built-in Decision Action, its outcome is recorded via durable execution when an Action State Store is configured, so recovery replays the verdict instead of asking the model again.

   What this could mean for the #1067 design:

   • Only the escalation branch needs durable suspension. The questions raised above (in-flight keys buffering the approval signal, watermark progression, an explicit persisted approval state machine) still need answers, but they would apply to a much smaller share of records, which may make a narrower first version acceptable.
   • If the pending-approval state records who resolved it (a decision model with its probabilities, or a human with an identity), the audit trail stays uniform across both paths.
   • Thresholds and the escalation policy should stay in application code, consistent with the DecisionModel design, where the business Action "chooses thresholds, and decides what to do with the answers".

   Happy to help write this up as a documented pattern or example, and to join the Discussion once it's opened.
3. Ashfaqbs commented on Oct 21, 2026

   

   Ashfaqbs

   on Oct 21, 2026

   ContributorAuthor

   More actions

   @dangzitou that is a good angle, thanks. Routing routine proposals through a decision model first and only escalating the ambiguous/high-impact ones to a human means the hard durable-suspend-resume problem only has to cover the escalation branch, which could make a first version a lot smaller and more acceptable to review.

   @wenjin272 you asked for a Discussion before implementation back in August - opening one now that folds in this pluggable-approver framing. Will link it here once it's up.

   👍React with 👍1Reacted by Wenjin Xie
4. Ashfaqbs commented on Oct 21, 2026

   

   Ashfaqbs

   on Oct 21, 2026

   ContributorAuthor

   More actions

   Discussion is up: #1304