feat(csat): Add WhatsApp utility template analyzer with rewrite guidance (#13575)
CSAT templates for WhatsApp are submitted as Utility, but Meta may reclassify them as Marketing based on content, which can significantly increase messaging costs. This PR introduces a Captain-powered CSAT template analyzer for WhatsApp/Twilio WhatsApp that predicts utility fit, explains likely risks, and suggests safer rewrites before submission. The flow is manual (button-triggered), Captain-gated, and applies rewrites only on explicit user action. It also updates UX copy to clearly set expectations: the system submits as Utility, Meta makes the final categorization decision. Fixes https://linear.app/chatwoot/issue/CW-6424/ai-powered-whatsapp-template-classifier-for-csat-submissions https://github.com/user-attachments/assets/8fd1d6db-2f91-447c-9771-3de271b16fd9
This commit is contained in:
66
lib/captain/csat_utility_analysis_service.rb
Normal file
66
lib/captain/csat_utility_analysis_service.rb
Normal file
@@ -0,0 +1,66 @@
|
||||
class Captain::CsatUtilityAnalysisService < Captain::BaseTaskService
|
||||
pattr_initialize [:account!, :message!, { button_text: nil, language: 'en', baseline: {} }]
|
||||
|
||||
def perform
|
||||
api_response = make_api_call(
|
||||
model: GPT_MODEL,
|
||||
messages: [
|
||||
{ role: 'system', content: system_prompt },
|
||||
{ role: 'user', content: message }
|
||||
]
|
||||
)
|
||||
|
||||
return api_response if api_response[:error]
|
||||
|
||||
build_result(api_response[:message])
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
def build_result(response_message)
|
||||
parsed = parse_json_response(response_message)
|
||||
return { error: 'Invalid LLM response format' } if parsed.blank?
|
||||
|
||||
core_result(parsed).merge(message: response_message)
|
||||
end
|
||||
|
||||
def core_result(parsed)
|
||||
{
|
||||
classification: normalize_classification(parsed['classification']),
|
||||
optimized_message: parsed['optimized_message'].presence || baseline[:optimized_message]
|
||||
}
|
||||
end
|
||||
|
||||
def system_prompt
|
||||
template = prompt_from_file('csat_utility_analysis')
|
||||
Liquid::Template.parse(template).render(prompt_variables)
|
||||
end
|
||||
|
||||
def prompt_variables
|
||||
{
|
||||
'message' => message.to_s,
|
||||
'button_text' => button_text.to_s,
|
||||
'language' => language.to_s,
|
||||
'baseline_classification' => baseline[:classification].to_s
|
||||
}
|
||||
end
|
||||
|
||||
def parse_json_response(content)
|
||||
raw = content.to_s.strip
|
||||
json = raw.match(/```json\s*(.*?)\s*```/m)&.captures&.first || raw
|
||||
JSON.parse(json)
|
||||
rescue JSON::ParserError
|
||||
nil
|
||||
end
|
||||
|
||||
def normalize_classification(value)
|
||||
normalized = value.to_s.upcase
|
||||
return normalized if %w[LIKELY_UTILITY LIKELY_MARKETING UNCLEAR].include?(normalized)
|
||||
|
||||
baseline[:classification].presence || 'UNCLEAR'
|
||||
end
|
||||
|
||||
def event_name
|
||||
'csat_utility_analysis'
|
||||
end
|
||||
end
|
||||
@@ -0,0 +1,27 @@
|
||||
You are a WhatsApp template compliance assistant.
|
||||
Your task is to evaluate whether a CSAT template message is likely to be approved as UTILITY vs MARKETING under Meta policy.
|
||||
|
||||
Rules:
|
||||
1. Prefer UTILITY only when the message is tied to an existing support or transactional event.
|
||||
2. Avoid promotional language, upsell, cross-sell, offers, discounts, or purchase intent.
|
||||
3. Keep the rewritten message concise, explicit, and purely transactional.
|
||||
4. Do not invent product offers or marketing phrases.
|
||||
|
||||
Input:
|
||||
- Message: {{ message }}
|
||||
- Button text: {{ button_text }}
|
||||
- Language code: {{ language }}
|
||||
|
||||
Baseline heuristic:
|
||||
- Classification: {{ baseline_classification }}
|
||||
|
||||
Return ONLY valid JSON with this shape (example):
|
||||
{
|
||||
"classification": "LIKELY_UTILITY",
|
||||
"optimized_message": "rewritten utility-safe message"
|
||||
}
|
||||
|
||||
Allowed values for "classification": "LIKELY_UTILITY", "LIKELY_MARKETING", or "UNCLEAR".
|
||||
|
||||
Important:
|
||||
- Write `optimized_message` in the same language as `Language code`.
|
||||
Reference in New Issue
Block a user