fix(captain): display handoff message to customer in V2 flow (#13885)
HandoffTool changes conversation status but only posts a private note.
ResponseBuilderJob now detects the tool flag and creates the public
handoff message that was previously only shown in V1.
# Pull Request Template
## Description
Captain V2 was silently forwarding conversations to humans without
showing a handoff message to the customer. The conversation appeared to
just stop
responding.
Root cause: In V2, HandoffTool calls bot_handoff! during agent
execution, which changes conversation status from pending to open. By
the time control returns
to ResponseBuilderJob#process_response, the conversation_pending? guard
returns early - skipping create_handoff_message entirely. The V1 flow
didn't have this
problem because AssistantChatService just returns a string token
(conversation_handoff) and lets ResponseBuilderJob handle everything.
What changed:
1. AgentRunnerService now surfaces the handoff_tool_called flag (already
tracked internally for usage metadata) in its response hash.
2. ResponseBuilderJob#handoff_requested? detects handoffs from both V1
(response token) and V2 (tool flag).
3. ResponseBuilderJob#process_response checks handoff_requested? before
the conversation_pending? guard, so V2 handoffs are processed even when
the status has
already changed.
4. ResponseBuilderJob#process_action('handoff') captures
conversation_pending? before calling bot_handoff! and uses that snapshot
to guard both bot_handoff!
and the OOO message - preventing double-execution when V2's HandoffTool
already ran them.
New V2 handoff flow:
AgentRunnerService
→ agent calls HandoffTool (creates private note, calls bot_handoff!)
→ returns response with handoff_tool_called: true
ResponseBuilderJob#process_response
→ handoff_requested? detects the flag
→ process_action('handoff')
→ create_handoff_message (public message for customer)
→ bot_handoff! skipped (conversation_pending? is false)
→ OOO skipped (conversation_pending? is false)
Fixes #13881
## Type of change
Please delete options that are not relevant.
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality not to work as expected)
- [ ] This change requires a documentation update
## How Has This Been Tested?
- Update existing response_builder_job_spec.rb covering the V2 handoff
path, V2 normal response path, and V1 regression
- Updated existing agent_runner_service_spec.rb expectations for the new
handoff_tool_called key and added a context for when the flag is true
## Checklist:
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] Any dependent changes have been merged and published in downstream
modules
---------
Co-authored-by: Aakash Bakhle <48802744+aakashb95@users.noreply.github.com>
Co-authored-by: aakashb95 <aakashbakhle@gmail.com>
This commit is contained in:
zip-fa
@@ -45,11 +45,28 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
|
||||
end
|
||||
|
||||
def process_response
|
||||
return unless conversation_pending?
|
||||
# Check V2 before V1: error_response can set both signals at once when HandoffTool
|
||||
# fired before the runner errored. V2 must win — running V1 on top would duplicate
|
||||
# OOO and re-dispatch the bot_handoff event.
|
||||
if v2_handoff_tool_fired?
|
||||
if conversation_pending?
|
||||
# HandoffTool flipped the flag without committing — its perform returned a
|
||||
# failure string (e.g. "Conversation not found") before bot_handoff! ran. Fall
|
||||
# back to a full V1 handoff so the customer still ends up with a human.
|
||||
process_v1_handoff
|
||||
else
|
||||
# HandoffTool already opened the conversation inside the agent loop. All that's
|
||||
# left is the customer-facing follow-up message.
|
||||
process_v2_handoff
|
||||
end
|
||||
elsif v1_handoff_requested?
|
||||
# V1 only signals via the response string — no state has been touched yet. If
|
||||
# the conversation isn't pending anymore, a human took over mid-run; bail out
|
||||
# rather than posting a stale handoff message on top of their reply.
|
||||
return unless conversation_pending?
|
||||
|
||||
if handoff_requested?
|
||||
process_action('handoff')
|
||||
else
|
||||
process_v1_handoff
|
||||
elsif conversation_pending?
|
||||
ActiveRecord::Base.transaction do
|
||||
create_messages
|
||||
Rails.logger.info("[CAPTAIN][ResponseBuilderJob] Incrementing response usage for #{account.id}")
|
||||
@@ -84,18 +101,27 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
|
||||
Captain::OpenAiMessageBuilderService.new(message: message).generate_content
|
||||
end
|
||||
|
||||
def handoff_requested?
|
||||
def v1_handoff_requested?
|
||||
@response['response'] == 'conversation_handoff'
|
||||
end
|
||||
|
||||
def process_action(action)
|
||||
case action
|
||||
when 'handoff'
|
||||
I18n.with_locale(@assistant.account.locale) do
|
||||
create_handoff_message
|
||||
@conversation.bot_handoff!
|
||||
send_out_of_office_message_if_applicable
|
||||
end
|
||||
def v2_handoff_tool_fired?
|
||||
@response['handoff_tool_called']
|
||||
end
|
||||
|
||||
def process_v1_handoff
|
||||
I18n.with_locale(@assistant.account.locale) do
|
||||
create_handoff_message
|
||||
@conversation.bot_handoff!
|
||||
send_out_of_office_message_if_applicable
|
||||
end
|
||||
end
|
||||
|
||||
def process_v2_handoff
|
||||
# HandoffTool already ran bot_handoff! + OOO inside the agent loop. Preserve
|
||||
# waiting_since so this message doesn't clear the timestamp it left in place.
|
||||
I18n.with_locale(@assistant.account.locale) do
|
||||
create_handoff_message(preserve_waiting_since: true)
|
||||
end
|
||||
end
|
||||
|
||||
@@ -107,9 +133,10 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
|
||||
::MessageTemplates::Template::OutOfOffice.perform_if_applicable(@conversation)
|
||||
end
|
||||
|
||||
def create_handoff_message
|
||||
def create_handoff_message(preserve_waiting_since: false)
|
||||
create_outgoing_message(
|
||||
@assistant.config['handoff_message'].presence || I18n.t('conversations.captain.handoff')
|
||||
@assistant.config['handoff_message'].presence || I18n.t('conversations.captain.handoff'),
|
||||
preserve_waiting_since: preserve_waiting_since
|
||||
)
|
||||
end
|
||||
|
||||
@@ -122,7 +149,7 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
|
||||
raise ArgumentError, 'Message content cannot be blank' if content.blank?
|
||||
end
|
||||
|
||||
def create_outgoing_message(message_content, agent_name: nil)
|
||||
def create_outgoing_message(message_content, agent_name: nil, preserve_waiting_since: false)
|
||||
additional_attrs = {}
|
||||
additional_attrs[:agent_name] = agent_name if agent_name.present?
|
||||
|
||||
@@ -132,13 +159,14 @@ class Captain::Conversation::ResponseBuilderJob < ApplicationJob
|
||||
inbox_id: inbox.id,
|
||||
sender: @assistant,
|
||||
content: message_content,
|
||||
additional_attributes: additional_attrs
|
||||
additional_attributes: additional_attrs,
|
||||
preserve_waiting_since: preserve_waiting_since
|
||||
)
|
||||
end
|
||||
|
||||
def handle_error(error)
|
||||
log_error(error)
|
||||
process_action('handoff') if conversation_pending?
|
||||
process_v1_handoff if conversation_pending?
|
||||
true
|
||||
end
|
||||
|
||||
|
||||
@@ -24,6 +24,7 @@ class Captain::Assistant::AgentRunnerService
|
||||
@conversation = conversation
|
||||
@callbacks = callbacks
|
||||
@source = source
|
||||
@handoff_tool_called = false
|
||||
end
|
||||
|
||||
def generate_response(message_history: [])
|
||||
@@ -98,13 +99,15 @@ class Captain::Assistant::AgentRunnerService
|
||||
output = result.output
|
||||
response = output.is_a?(Hash) ? output.with_indifferent_access : { 'response' => output.to_s, 'reasoning' => 'Processed by agent' }
|
||||
response['agent_name'] = result.context&.dig(:current_agent)
|
||||
response['handoff_tool_called'] = result.context&.dig(:captain_v2_handoff_tool_called) || false
|
||||
response
|
||||
end
|
||||
|
||||
def error_response(error_message)
|
||||
{
|
||||
'response' => 'conversation_handoff',
|
||||
'reasoning' => "Error occurred: #{error_message}"
|
||||
'reasoning' => "Error occurred: #{error_message}",
|
||||
'handoff_tool_called' => @handoff_tool_called
|
||||
}
|
||||
end
|
||||
|
||||
@@ -175,16 +178,18 @@ class Captain::Assistant::AgentRunnerService
|
||||
end
|
||||
|
||||
def add_usage_metadata_callback(runner)
|
||||
return runner unless ChatwootApp.otel_enabled?
|
||||
|
||||
handoff_tool_name = Captain::Tools::HandoffTool.new(@assistant).name
|
||||
|
||||
# Tool tracking always runs — process_response in the job consumes the resulting
|
||||
# handoff_tool_called flag regardless of whether OTEL is enabled.
|
||||
runner.on_tool_complete do |tool_name, _tool_result, context_wrapper|
|
||||
track_handoff_usage(tool_name, handoff_tool_name, context_wrapper)
|
||||
end
|
||||
|
||||
runner.on_run_complete do |_agent_name, _result, context_wrapper|
|
||||
write_credits_used_metadata(context_wrapper)
|
||||
if ChatwootApp.otel_enabled?
|
||||
runner.on_run_complete do |_agent_name, _result, context_wrapper|
|
||||
write_credits_used_metadata(context_wrapper)
|
||||
end
|
||||
end
|
||||
runner
|
||||
end
|
||||
@@ -193,15 +198,17 @@ class Captain::Assistant::AgentRunnerService
|
||||
return unless context_wrapper&.context
|
||||
return unless tool_name.to_s == handoff_tool_name
|
||||
|
||||
# Mirror the flag onto the instance so error_response can surface it even when
|
||||
# the runner raises before returning a result (the context is unreachable then).
|
||||
context_wrapper.context[:captain_v2_handoff_tool_called] = true
|
||||
@handoff_tool_called = true
|
||||
end
|
||||
|
||||
def write_credits_used_metadata(context_wrapper)
|
||||
root_span = context_wrapper&.context&.dig(:__otel_tracing, :root_span)
|
||||
return unless root_span
|
||||
|
||||
credit_used = !context_wrapper.context[:captain_v2_handoff_tool_called]
|
||||
root_span.set_attribute(format(ATTR_LANGFUSE_METADATA, 'credit_used'), credit_used.to_s)
|
||||
root_span.set_attribute(format(ATTR_LANGFUSE_METADATA, 'credit_used'), @handoff_tool_called ? 'false' : 'true')
|
||||
end
|
||||
|
||||
def runner
|
||||
|
||||
Reference in New Issue
Block a user