fix(captain): display handoff message to customer in V2 flow (#13885)
HandoffTool changes conversation status but only posts a private note.
ResponseBuilderJob now detects the tool flag and creates the public
handoff message that was previously only shown in V1.
# Pull Request Template
## Description
Captain V2 was silently forwarding conversations to humans without
showing a handoff message to the customer. The conversation appeared to
just stop
responding.
Root cause: In V2, HandoffTool calls bot_handoff! during agent
execution, which changes conversation status from pending to open. By
the time control returns
to ResponseBuilderJob#process_response, the conversation_pending? guard
returns early - skipping create_handoff_message entirely. The V1 flow
didn't have this
problem because AssistantChatService just returns a string token
(conversation_handoff) and lets ResponseBuilderJob handle everything.
What changed:
1. AgentRunnerService now surfaces the handoff_tool_called flag (already
tracked internally for usage metadata) in its response hash.
2. ResponseBuilderJob#handoff_requested? detects handoffs from both V1
(response token) and V2 (tool flag).
3. ResponseBuilderJob#process_response checks handoff_requested? before
the conversation_pending? guard, so V2 handoffs are processed even when
the status has
already changed.
4. ResponseBuilderJob#process_action('handoff') captures
conversation_pending? before calling bot_handoff! and uses that snapshot
to guard both bot_handoff!
and the OOO message - preventing double-execution when V2's HandoffTool
already ran them.
New V2 handoff flow:
AgentRunnerService
→ agent calls HandoffTool (creates private note, calls bot_handoff!)
→ returns response with handoff_tool_called: true
ResponseBuilderJob#process_response
→ handoff_requested? detects the flag
→ process_action('handoff')
→ create_handoff_message (public message for customer)
→ bot_handoff! skipped (conversation_pending? is false)
→ OOO skipped (conversation_pending? is false)
Fixes #13881
## Type of change
Please delete options that are not relevant.
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How Has This Been Tested?
- Update existing response_builder_job_spec.rb covering the V2 handoff
path, V2 normal response path, and V1 regression
- Updated existing agent_runner_service_spec.rb expectations for the new
handoff_tool_called key and added a context for when the flag is true
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
This commit is contained in:
@@ -24,6 +24,7 @@ class Captain::Assistant::AgentRunnerService
|
||||
@conversation = conversation
|
||||
@callbacks = callbacks
|
||||
@source = source
|
||||
@handoff_tool_called = false
|
||||
end
|
||||
|
||||
def generate_response(message_history: [])
|
||||
@@ -98,13 +99,15 @@ class Captain::Assistant::AgentRunnerService
|
||||
output = result.output
|
||||
response = output.is_a?(Hash) ? output.with_indifferent_access : { 'response' => output.to_s, 'reasoning' => 'Processed by agent' }
|
||||
response['agent_name'] = result.context&.dig(:current_agent)
|
||||
response['handoff_tool_called'] = result.context&.dig(:captain_v2_handoff_tool_called) || false
|
||||
response
|
||||
end
|
||||
|
||||
def error_response(error_message)
|
||||
{
|
||||
'response' => 'conversation_handoff',
|
||||
'reasoning' => "Error occurred: #{error_message}"
|
||||
'reasoning' => "Error occurred: #{error_message}",
|
||||
'handoff_tool_called' => @handoff_tool_called
|
||||
}
|
||||
end
|
||||
|
||||
@@ -175,16 +178,18 @@ class Captain::Assistant::AgentRunnerService
|
||||
end
|
||||
|
||||
def add_usage_metadata_callback(runner)
|
||||
return runner unless ChatwootApp.otel_enabled?
|
||||
|
||||
handoff_tool_name = Captain::Tools::HandoffTool.new(@assistant).name
|
||||
|
||||
# Tool tracking always runs — process_response in the job consumes the resulting
|
||||
# handoff_tool_called flag regardless of whether OTEL is enabled.
|
||||
runner.on_tool_complete do |tool_name, _tool_result, context_wrapper|
|
||||
track_handoff_usage(tool_name, handoff_tool_name, context_wrapper)
|
||||
end
|
||||
|
||||
runner.on_run_complete do |_agent_name, _result, context_wrapper|
|
||||
write_credits_used_metadata(context_wrapper)
|
||||
if ChatwootApp.otel_enabled?
|
||||
runner.on_run_complete do |_agent_name, _result, context_wrapper|
|
||||
write_credits_used_metadata(context_wrapper)
|
||||
end
|
||||
end
|
||||
runner
|
||||
end
|
||||
@@ -193,15 +198,17 @@ class Captain::Assistant::AgentRunnerService
|
||||
return unless context_wrapper&.context
|
||||
return unless tool_name.to_s == handoff_tool_name
|
||||
|
||||
# Mirror the flag onto the instance so error_response can surface it even when
|
||||
# the runner raises before returning a result (the context is unreachable then).
|
||||
context_wrapper.context[:captain_v2_handoff_tool_called] = true
|
||||
@handoff_tool_called = true
|
||||
end
|
||||
|
||||
def write_credits_used_metadata(context_wrapper)
|
||||
root_span = context_wrapper&.context&.dig(:__otel_tracing, :root_span)
|
||||
return unless root_span
|
||||
|
||||
credit_used = !context_wrapper.context[:captain_v2_handoff_tool_called]
|
||||
root_span.set_attribute(format(ATTR_LANGFUSE_METADATA, 'credit_used'), credit_used.to_s)
|
||||
root_span.set_attribute(format(ATTR_LANGFUSE_METADATA, 'credit_used'), @handoff_tool_called ? 'false' : 'true')
|
||||
end
|
||||
|
||||
def runner
|
||||
|
||||
Reference in New Issue
Block a user