Commit Graph

89 Commits

Author SHA1 Message Date
Muhsin Keloth
598ece9a2d fix: Handle Facebook reel attachment type (#13691)
Fixes https://linear.app/chatwoot/issue/PLA-96/argumenterror-reel-is-not-a-valid-file-type-argumenterror
Fixes a crash in `when a user shares a Facebook reel via Messenger.
Facebook sends these attachments with `type: "reel"`, which isn't a
valid `file_type` enum value on the Attachment model (only `ig_reel`
exists, matching Instagram's format).

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2026-03-06 08:49:41 +04:00
Vinay Keerthi
059506b1db feat: Add automatic favicon fetching for companies (#13013)
## Summary

This Enterprise-only feature automatically fetches a favicon for
companies created with a domain, and adds a batch task to backfill
missing avatars for existing companies. The flow only targets companies
that do not already have an attached avatar, so existing avatars are
left untouched.


## Demo 



https://github.com/user-attachments/assets/d050334e-769f-4e46-b6e7-f7423727a192



## What changed

- Added `Avatar::AvatarFromFaviconJob` to build a Google favicon URL
from the company domain and fetch it through `Avatar::AvatarFromUrlJob`
- Triggered favicon fetching from `Company` with `after_create_commit`
- Added `Companies::FetchAvatarsJob` to batch existing companies that
are missing avatars
- Added `companies:fetch_missing_avatars` under `enterprise/lib/tasks`
- Kept the company-specific implementation inside the Enterprise
boundary
- Stubbed the new favicon request in unrelated specs that now hit this
callback indirectly
- Updated a couple of CI-sensitive specs that were failing due to
callback side effects / reload-safe exception assertions

## How to verify

1. Create a company in Enterprise with a valid domain and no avatar.
2. Confirm that a favicon-based avatar gets attached shortly after
creation.
3. Create another company with a domain and an avatar already attached.
4. Confirm that the existing avatar is not replaced.
5. Run `companies:fetch_missing_avatars`.
6. Confirm that existing companies without avatars get one, while
companies that already have avatars remain unchanged.

## Notes

- This change does not refresh or overwrite existing company avatars
- Favicon fetching only runs for companies with a present domain
- The branch includes the latest `develop`

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2026-03-05 18:51:28 -08:00
Muhsin Keloth
bdcc62f1b0 feat(facebook): Mark Messenger native-app echoes as external echo message (#13665)
When agents send replies from the native Facebook Messenger app (not
Chatwoot), echo events were created without external_echo metadata and
could be misrepresented in the UI. This change updates Messenger echo
message creation to:

- set content_attributes.external_echo = true for outgoing_echo messages
- set echo message status to delivered
- keep sender as nil for echo messages (existing behavior)

<img width="2614" height="1264" alt="CleanShot 2026-02-26 at 16 32
04@2x"
src="https://github.com/user-attachments/assets/ba61c941-465d-4893-814e-855e6b6c79e8"
/>
2026-02-26 19:05:15 +04:00
Sojan Jose
9eb3ee44a8 Revert "chore: Upgrade Rails to 7.2.2 and update Gemfile dependencies (#11037)"
This reverts commit ef6ba8aabd.
2026-02-03 21:09:42 -08:00
Sojan Jose
ef6ba8aabd chore: Upgrade Rails to 7.2.2 and update Gemfile dependencies (#11037)
Upgrade rails to 7.2.2 so that we can proceed with the rails 8 upgrade
afterwards
 
 # Changelog
- `.circleci/config.yml` — align CI DB setup with GitHub Actions
(`db:create` + `db:schema:load`) to avoid trigger-dependent prep steps.
- `.rubocop.yml` — add `rubocop-rspec_rails` and disable new cops that
don't match existing spec style.
- `AGENTS.md` — document that specs should run without `.env` (rename
temporarily when present).
- `Gemfile` — upgrade to Rails 7.2, switch Azure storage gem, pin
`commonmarker`, bump `sidekiq-cron`, add `rubocop-rspec_rails`, and
relax some gem pins.
- `Gemfile.lock` — dependency lockfile updates from the Rails 7.2 and
gem changes.
- `app/controllers/api/v1/accounts/integrations/linear_controller.rb` —
stringify params before passing to the Linear service to keep key types
stable.
- `app/controllers/super_admin/instance_statuses_controller.rb` — use
`MigrationContext` API for migration status in Rails 7.2.
- `app/models/installation_config.rb` — add commentary on YAML
serialization and future JSONB migration (no behavior change).
- `app/models/integrations/hook.rb` — ensure hook type is set on create
only and guard against missing app.
- `app/models/user.rb` — update enum syntax for Rails 7.2 deprecation,
serialize OTP backup codes with JSON, and use Ruby `alias`.
- `app/services/crm/leadsquared/setup_service.rb` — stringify hook
settings keys before merge to keep JSON shape consistent.
- `app/services/macros/execution_service.rb` — remove macro-specific
assignee activity workaround; rely on standard assignment handlers.
- `config/application.rb` — load Rails 7.2 defaults.
- `config/storage.yml` — update Azure Active Storage service name to
`AzureBlob`.
- `db/migrate/20230515051424_update_article_image_keys.rb` — use
credentials `secret_key_base` with fallback to legacy secrets.
- `docker/Dockerfile` — add `yaml-dev` and `pkgconf` packages for native
extensions (Ruby 3.4 / psych).
- `lib/seeders/reports/message_creator.rb` — add parentheses for clarity
in range calculation.
- `package.json` — pin Vite version and bump `vite-plugin-ruby`.
- `pnpm-lock.yaml` — lockfile changes from JS dependency updates.
- `spec/builders/v2/report_builder_spec.rb` — disable transactional
fixtures; truncate tables per example via Rails `truncate_tables` so
after_commit callbacks run with clean isolation; keep builder spec
metadata minimal.
- `spec/builders/v2/reports/label_summary_builder_spec.rb` — disable
transactional fixtures + truncate tables via Rails `truncate_tables`;
revert to real `resolved!`/`open!`/`resolved!` flow for multiple
resolution events; align date range to `Time.zone` to avoid offset gaps;
keep builder spec metadata minimal.
- `spec/controllers/api/v1/accounts/macros_controller_spec.rb` — assert
`assignee_id` instead of activity message to avoid transaction-timing
flakes.
- `spec/services/telegram/incoming_message_service_spec.rb` — reference
the contact tied to the created conversation instead of
`Contact.all.first` to avoid order-dependent failures when other specs
leave data behind.
-
`spec/mailers/administrator_notifications/shared/smtp_config_shared.rb`
— use `with_modified_env` instead of stubbing mailer internals.
- `spec/services/account/sign_up_email_validation_service_spec.rb` —
compare error `class.name` for parallel/reload-safe assertions.
2026-02-03 14:29:26 -08:00
Pranav
5ec77aca64 feat: Add first response time distribution report endpoint (#13400)
The index is already added in production.

Adds a new reporting API that returns conversation counts grouped by
channel type and first response time buckets (0-1h, 1-4h, 4-8h, 8-24h,
24h+).

- GET /api/v2/accounts/:id/reports/first_response_time_distribution
- Uses SQL aggregation to handle large datasets efficiently
- Adds composite index on reporting_events for query performance

Tested on production workload.
Request: GET
`/api/v2/accounts/1/reports/first_response_time_distribution?since=<since>&until=<until>`
Response payload:
```
{
    "Channel::WebWidget": {
      "0-1h": 120,
      "1-4h": 85,
      "4-8h": 32,
      "8-24h": 12,
      "24h+": 3
    },
    "Channel::Email": {
      "0-1h": 12,
      "1-4h": 28,
      "4-8h": 45,
      "8-24h": 35,
      "24h+": 10
    },
    "Channel::FacebookPage": {
      "0-1h": 50,
      "1-4h": 30,
      "4-8h": 15,
      "8-24h": 8,
      "24h+": 2
    }
  }
```

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2026-01-30 22:22:27 +04:00
Muhsin Keloth
6f45af605c feat: Add inbox-label matrix report endpoint (#13394)
This PR added new API endpoint GET
/api/v2/accounts/:account_id/reports/inbox_label_matrix that returns
conversation counts grouped by inbox and label in a matrix format.
Supports optional filtering by date range, inbox_ids, and label_ids.

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2026-01-29 13:32:59 -08:00
Muhsin Keloth
aaeea6c9bf feat: Display story replies with attachment and context label (#13356)
Fixes https://github.com/chatwoot/chatwoot/issues/13354
Fixes
https://linear.app/chatwoot/issue/CW-6394/story-responses-are-not-being-shown-in-the-ui
When someone replies to your Instagram story, agents in Chatwoot only
see the reply text with no story image and no indication that it was a
story reply. This makes it impossible to understand what the customer is
responding to the message looks like a random text with no context. For
example, if a customer replies "Love this!" to your story, the agent
just sees "Love this!" with no way to know which story triggered the
conversation. This PR fixes the issue by storing the story attachment
and adding a context label.

<img width="1408" height="2052" alt="CleanShot 2026-01-27 at 19 19
38@2x"
src="https://github.com/user-attachments/assets/341afea9-98e3-4e47-b2fa-ef77fe32851f"
/>
2026-01-28 16:47:04 +04:00
Pranav
0917e1a646 feat: Add an API to support querying metrics by ChannelType (#13255)
This API gives you how many conversations exist per channel, broken down
by status in a given time period. The max time period is capped to 6
months for now.

**Input Params:**
- **since:** Unix timestamp (seconds) - start of date range
- **until:** Unix timestamp (seconds) - end of date range


**Response Payload:**

```json
{
  "Channel::Sms": {
    "resolved": 85,
    "snoozed": 10,
    "open": 5,
    "pending": 5,
    "total": 100
  },
  "Channel::Email": {
    "resolved": 72,
    "snoozed": 15,
    "open": 13,
    "pending": 13,
    "total": 100
  },
  "Channel::WebWidget": {
    "resolved": 90,
    "snoozed": 7,
    "open": 3,
    "pending": 3,
    "total": 100
  }
}
```

**Definitons:**
resolved = Number of conversations created within the selected time
period that are currently marked as resolved.
snoozed = Number of conversations created within the selected time
period that are currently marked as snoozed.
pending = Number of conversations created within the selected time
period that are currently marked as pending.
open = Number of conversations created within the selected time period
that are currently open.
total = Total number of conversations created within the selected time
period, across all statuses.
2026-01-12 23:18:47 -08:00
Pranav
bb8bafe3dc feat(ce): Add Year in review feature (#13078)
<img width="1502" height="813" alt="Screenshot 2025-12-15 at 5 01 57 PM"
src="https://github.com/user-attachments/assets/ea721f00-403c-4adc-8410-5c0fa4ee4122"
/>
2025-12-15 17:24:45 -08:00
Vishnu Narayanan
08b9134486 feat: speed up circleci and github actions (#12849)
# 🚀 Speed up CI/CD test execution with parallelization

## TL;DR

- **Problem**: CI tests took 36-42 minutes per commit, blocking
developer workflow
- **Solution**: Implemented 16-way parallelization + optimized slow
tests + fixed Docker builds
- **Impact**: **1,358 hours/month saved** (7.7 FTE) across GitHub
Actions + CircleCI
  - GitHub Actions tests: 36m → 7m (82% faster)
  - Backend tests: 28m → 4m (87% faster with 16-way parallelization)
  - CircleCI tests: 42m → 7m (83% faster)
  - Docker builds: 34m → 5m (85% faster)
- **Result**: 5-6x faster feedback loops, 100% success rate on recent
runs

---

## Problem
CI test runs were taking **36-42 minutes per commit push** (GitHub
Actions: 36m avg, CircleCI: 42m P95), creating a significant bottleneck
in the development workflow.

## Solution
This PR comprehensively restructures both CI test pipelines to leverage
16-way parallelization and optimize test execution, reducing test
runtime from **36-42 minutes to ~7 minutes** - an **82% improvement**.

---

## 📊 Real Performance Data (Both CI Systems)

### GitHub Actions

#### Before (develop branch - 5 recent runs)
```
Individual runs: 35m 29s | 36m 1s | 40m 0s | 36m 4s | 34m 18s
Average: 36m 22s
```

#### After (feat/speed_up_ci branch - 9 successful runs)
```
Individual runs: 6m 39s | 7m 2s | 6m 53s | 6m 26s | 6m 52s | 6m 42s | 6m 45s | 6m 40s | 6m 37s
Average: 6m 44s
Range: 6m 26s - 7m 2s
```

**Improvement**:  **81.5% faster** (29m 38s saved per run)


#### Backend Tests Specific Impact
With 16-way parallelization, backend tests show dramatic improvement:
- **Before**: 27m 52s (sequential execution)
- **After**: 3m 44s (longest of 16 parallel runners)
  - Average across runners: 2m 30s
  - Range: 1m 52s - 3m 44s
- **Improvement**:  **86.6% faster** (24m 8s saved)
---

### CircleCI

#### Before (develop branch - CircleCI Insights)
```
Duration (P95): 41m 44s
Runs: 70 (last 30 days)
Success Rate: 84%
```

#### After (feat/speed_up_ci branch - Last 2 pipeline runs)
```
Run 1 (1h ago): 7m 7s
  ├─ lint: 4m 12s
  ├─ frontend-tests: 5m 36s
  ├─ backend-tests: 6m 23s
  ├─ coverage: 20s
  └─ build: 1s

Run 2 (2h ago): 7m 21s
  ├─ lint: 3m 47s
  ├─ frontend-tests: 5m 4s
  ├─ backend-tests: 6m 33s
  ├─ coverage: 19s
  └─ build: 1s

Average: 7m 14s
Success Rate: 100% 
```

**Improvement**:  **82.7% faster** (34m 30s saved per run)

---

## 🐳 Related Work: Docker Build Optimization

As part of the broader CI/CD optimization effort, Docker build
performance was improved separately in **PR #12859**.

### Docker Build Fix (Merged Separately)
**Problem**: Multi-architecture Docker builds (amd64/arm64) were taking
~34 minutes due to cache thrashing

**Solution**: Added separate cache scopes per platform in
`.github/workflows/test_docker_build.yml`:
```yaml
cache-from: type=gha,scope=${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=${{ matrix.platform }}
```

**Results** (measured from November 2025 data):
- **Before**: 34.2 minutes/run average (15,547 minutes across 454 runs)
- **After**: 5 minutes/run
- **Improvement**: 85% faster, 29.2 minutes saved per run
- **Frequency**: 25.2 runs/day
- **Monthly savings**: **369 hours** (46 developer-days)

This prevents different architectures from invalidating each other's
caches and contributes 27% of total CI/CD time savings.

---

## 🎯 Key Findings

### Both CI Systems Now Perform Similarly
- **CircleCI**: 7m 14s average
- **GitHub Actions**: 6m 44s average
- **Difference**: Only 30 seconds apart (remarkably consistent!)

### Combined Performance
- **Average improvement across both systems**: **82.1% faster**
- **Time saved per commit**: ~32 minutes
- **Developer feedback loop**: 36-42 minutes → ~7 minutes

### Success Rate Improvement
- **CircleCI**: 84% → 100% (on feat/speed_up_ci branch)
- **GitHub Actions**: 100% (all 9 recent runs successful)
- Fixed all test isolation issues that caused intermittent failures

### Impact at Scale (Based on Real November 2025 Data)
- **CI runs per day**: **30.8 average** for tests, **25.2** for Docker
builds
  - Measured from GitHub Actions Usage Metrics (18 days)
  - Weekdays: 38-54 runs/day
  - Peak: up to 68 runs in a single day
- **This PR (test suite only)**:
  - **Daily time saved**: **15.3 hours** (GitHub Actions + CircleCI)
- **Monthly time saved**: **458 hours** (57 developer-days) on GitHub
Actions
  - Additional **531 hours** (66 developer-days) on CircleCI
- **Combined with Docker optimization** (PR #12859): **1,358
hours/month** (see Summary)
- **Developer experience**: 5-6x faster iteration cycles

---

## Code Changes

### 1. **Backend Test Parallelization (16x)**
Both CI systems now use 16-way parallelization with identical
round-robin test distribution:

```bash
# Distribute tests evenly across 16 runners
SPEC_FILES=($(find spec -name '*_spec.rb' | sort))
for i in "${!SPEC_FILES[@]}"; do
  if [ $(( i % 16 )) -eq $RUNNER_INDEX ]; then
    TESTS="$TESTS ${SPEC_FILES[$i]}"
  fi
done
```

**Why round-robin over timing-based?**
- CircleCI's timing-based splitting grouped similar tests together
- This caused race conditions with OAuth callback tests (Linear,
Shopify, Notion)
- Round-robin ensures even distribution and better test isolation
- Both CI systems now behave identically

### 2. **Frontend Test Optimization**
Enabled Vitest thread parallelization in `vite.config.ts`:

```typescript
pool: 'threads',
poolOptions: {
  threads: {
    singleThread: false,
  },
},
```

### 3. **CI Architecture Restructuring**
Split monolithic CI jobs into parallel stages:
- **Lint** (backend + frontend) - runs independently for fast feedback
- **Frontend tests** - runs in parallel with backend
- **Backend tests** - 16-way parallelized across runners
- **Coverage** - aggregates results from test jobs
- **Build** (CircleCI only) - final job for GitHub status check
compatibility

### 4. **Critical Test Optimization**
**report_builder_spec.rb**: Changed `before` to `before_all`
- Reduced execution time from **19 minutes to 1.2 minutes** (16x
speedup)
- Setup now runs once instead of 21 times
- Single biggest performance improvement after parallelization

---

## Test Stability Fixes (10 spec files)

Parallelization exposed latent test isolation issues that were fixed:

### Object Identity Comparisons (6 files)
Tests were comparing Ruby object instances instead of IDs:
- `spec/models/integrations/hook_spec.rb` - Use `.pluck(:id)` for
comparisons
- `spec/enterprise/models/captain/scenario_spec.rb` - Compare IDs
instead of objects
- `spec/models/notification_spec.rb` - Compare IDs for sort order
validation
- `spec/models/account_spec.rb` - Compare IDs in scope queries
- `spec/services/widget/token_service_spec.rb` - Compare class names
instead of class objects
- `spec/models/concerns/avatarable_shared.rb` - Use `respond_to` checks
for ActiveStorage

### Database Query Caching
- `spec/jobs/delete_object_job_spec.rb` - Added `.reload` to force fresh
database queries

### Test Expectations Timing
- `spec/jobs/mutex_application_job_spec.rb` - Removed flaky unlock
expectation after error block
  - Related to original PR #8770
- Expectation after error block never executes in parallel environments

### Timezone Handling
- `spec/mailers/account_notification_mailer_spec.rb` - Fixed date
parsing at timezone boundaries
  - Changed test time from 23:59:59Z to 12:00:00Z

### Test Setup
- `spec/builders/v2/report_builder_spec.rb` - Optimized with
`before_all`

---

##  CircleCI GitHub Integration Fix

### Problem
GitHub PR checks were stuck on "Waiting for status" even when all
CircleCI jobs passed. GitHub was expecting a job named `build` but the
workflow only had a workflow named "build".

### Solution
Added an explicit `build` job that runs after all other jobs:

```yaml
build:
  steps:
    - run:
        name: Legacy build aggregator
        command: echo "All main jobs passed"
  requires:
    - lint
    - coverage
```

This ensures GitHub's required status checks work correctly.


---

##  Testing & Validation

-  **GitHub Actions**: 9 successful runs, consistent 6m 26s - 7m 2s
runtime
-  **CircleCI**: 2 successful runs, consistent 7m 7s - 7m 21s runtime
-  Both CI systems produce identical, consistent results
-  GitHub PR status checks complete correctly
-  Success rate improved from 84% to 100% (recent runs)
-  No test regressions introduced
-  All flaky tests fixed (callback controllers, mutex jobs, etc.)

---

## 🎉 Summary

This PR delivers an **82% improvement** in test execution time across
both CI systems:

- **GitHub Actions tests**: 36m → 7m (81.5% faster)
- Backend tests specifically: 28m → 4m (86.6% faster)
- **CircleCI tests**: 42m → 7m (82.7% faster)
- **Developer feedback loop**: 5-6x faster
- **Test stability**: 84% → 100% success rate

### 📊 Total CI/CD Impact (All Optimizations)

Based on real November 2025 data, combining this PR with Docker build
optimization (PR #12859):

**Monthly Time Savings**: **1,358 hours/month** = **170
developer-days/month** = **7.7 FTE**

| System | Runs/Day | Before | After | Savings | Monthly Impact |
|--------|----------|---------|--------|---------|----------------|
| **GitHub Actions Tests** | 30.8 | 36.5m | 6.7m | 29.8m/run | 458 hrs
(34%) |
| **GitHub Actions Docker** | 25.2 | 34.2m | 5.0m | 29.2m/run | 369 hrs
(27%) |
| **CircleCI Tests** | 30.8 | 41.7m | 7.2m | 34.5m/run | 531 hrs (39%) |

*Data source: GitHub Actions Usage Metrics (November 2025, 18 days),
CircleCI Insights (30 days)*

The combined optimizations save the equivalent of **nearly 8 full-time
developers** worth of CI waiting time every month, significantly
improving developer velocity and reducing CI costs. All test isolation
issues exposed by parallelization have been fixed, ensuring reliable and
consistent results across both CI platforms.

woot woot !!!

---------
2025-11-19 15:32:48 +05:30
Sivin Varghese
e81152608d fix: Issue with processing variables in outgoing email content (#12799)
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Co-authored-by: Vinay Keerthi <11478411+stonecharioteer@users.noreply.github.com>
Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-11-10 20:50:02 +05:30
Shivam Mishra
ce400a36d7 feat: Always process email content (#12734)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-10-30 13:36:39 +05:30
Pranav
368d7c4608 feat: Add support for HTML emails in outgoing messages (#12662)
This PR adds sending custom HTML content in outgoing email messages
through Chatwoot's Email channels, while maintaining backward
compatibility with existing markdown rendering.

###  API Usage

**Endpoint:** `POST
/api/v1/accounts/{account_id}/conversations/{conversation_id}/messages`

```json
  {
   "content": "Fallback text content",
   "email_html_content": "<div><h1>Welcome!</h1><p>This is <strong>custom HTML</strong></p></div>"
  }
```

---------

Co-authored-by: Muhsin <muhsinkeramam@gmail.com>
2025-10-15 13:22:23 +05:30
Pranav
47bdb6d2bb feat: Clean up email configuration for from and reply to emails (#12453)
We first added conversation continuity for the live chat widget, and
then carried the same logic over to email channels.

The problem was that this added a reply+conversationUUID@domain.com as
the reply-to for emails, which was unnecessary. For email channels, the
reply-to can just be the channel’s own email address.

That extra layer made things more complex than it needed to be. In this
PR, I’ve cleaned up the config so it’s simpler. The table below shows
how it’ll work going forward.

---

| Type | From Email | Reply To Email |
| -- | -- | -- |
| Standard IMAP, SMTP email channel | channel.email | channel.email |
| Google OAuth Email channel | channel.email | channel.email |
| Microsoft OAuth Email channel | channel.email | channel.email |
| Email forwarded to Chatwoot, brought their own SMTP | channel.email |
channel.email |
| Imap to fetch email, Use Chatwoot's SMTP | channel.email if verified
with Chatwoot's SMTP provider. Otherwise account support email |
channel.email |
| Email forwarded to Chatwoot, Use Chatwoot's SMTP | channel.email if
verified with Chatwoot's SMTP provider. Otherwise account support email
| channel.email |
| -- | --  | -- |
| Website Live Chat - Conversation Continuity Inbound Emails enabled|
Account Support Email | reply+{conversation-uuid}@{account_domain} |
| Website Live Chat - Conversation Continuity Inbound Emails disabled|
Account Support Email | Account Support Email |

Fixes https://github.com/chatwoot/chatwoot/issues/10614
Fixes https://github.com/chatwoot/chatwoot/issues/10521
Fixes https://github.com/chatwoot/chatwoot/issues/10300
Fixes https://github.com/chatwoot/chatwoot/issues/10091
Fixes https://github.com/chatwoot/chatwoot/issues/4890
Fixes https://github.com/chatwoot/chatwoot/issues/10676
Fixes https://github.com/chatwoot/chatwoot/issues/10756
Fixes https://github.com/chatwoot/chatwoot/issues/11515
Fixes https://github.com/chatwoot/chatwoot/issues/9471

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-09-24 11:36:53 -07:00
Shivam Mishra
76c110e60e fix: resolution count does not have account scope (#12370) 2025-09-04 18:04:00 +05:30
Shivam Mishra
ef4e287f0d fix: wrong resolution count in timeseries reports (#12261)
There was a fundamental difference in how resolution counts were
calculated between the agent summary and timeseries reports, causing
confusion for users when the numbers didn't match.

The agent summary report counted all `conversation_resolved` events
within a time period by querying the `reporting_events` table directly.
However, the timeseries report had an additional constraint that
required the conversation to currently be in resolved status
(`conversations.status = 1`). This meant that if an agent resolved a
conversation that was later reopened, the resolution action would be
counted in the summary but not in the timeseries.

This fix aligns both reports to count resolution events rather than
conversations in resolved state. When an agent resolves a conversation,
they should receive credit for that action regardless of what happens to
the conversation afterward. The same logic now applies to bot
resolutions as well.

The change removes the `conversations: { status: :resolved }` condition
from both `scope_for_resolutions_count` and
`scope_for_bot_resolutions_count` methods in CountReportBuilder, and
updates the corresponding test expectations to reflect that all
resolution events are counted.


## About timezone

When a timezone is specified via `timezone_offset` parameter, the
reporting system:

1. Converts timestamps to the target timezone before grouping
2. Groups data by local day/week/month boundaries in that timezone, but
the primary boundaries are sent by the frontend and used as-is
3. Returns timestamps representing midnight in the target timezone

This means the same events can appear in different day buckets depending
on the timezone used. For summary reports, it works fine, since the user
only needs the total count between two timestamps and the frontend sends
the timestamps adjusted for timezone.

## Testing Locally

Run the following command, this will erase all data for that account and
put in 1000 conversations over last 3 months, parameters of this can be
tweaked in `Seeders::Reports::ReportDataSeeder`

I'd suggest updating the values to generate data over 30 days, with
10000 conversations, it will take it's sweet time to run but then the
data will be really rich, great for testing.

```
ACCOUNT_ID=2 ENABLE_ACCOUNT_SEEDING=true bundle exec rake db:seed:reports_data
```

Pro Tip: Don't run the app when the seeder is active, we manually create
the reporting events anyway. So once done just use `redis-cli FLUSHALL`
to clear all sidekiq jobs. Will be easier on the system

Use the following scripts to test it

- https://gist.github.com/scmmishra/1263a922f5efd24df8e448a816a06257
- https://gist.github.com/scmmishra/ca0b861fa0139e2cccdb72526ea844b2
- https://gist.github.com/scmmishra/5fe73d1f48f35422fd1fd142ea3498f3
- https://gist.github.com/scmmishra/3b7b1f9e2ff149007170e5c329432f45
- https://gist.github.com/scmmishra/f245fa2f44cd973e5d60aac64f979162

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-09-03 15:47:16 +05:30
Sojan Jose
c31325e982 fix: resolve mutex conflicts in Instagram webhook specs (#12154)
This PR fixes flaky test failures in the Instagram webhook specs that
were caused by Redis mutex lock conflicts when
   tests ran in parallel.

 ### The Problem:
The InstagramEventsJob uses a Redis mutex with a key based on sender_id
and ig_account_id to prevent race
conditions. However, all test factories were using the same hardcoded
sender_id: 'Sender-id-1', causing multiple
test instances to compete for the same mutex lock when running in
parallel.

 ### The Solution:
- Updated all Instagram event factories to generate unique sender IDs
using SecureRandom.hex(4)
- Modified test stubs and expectations to work with dynamic sender IDs
instead of hardcoded values
- Ensured each test instance gets its own unique mutex key, eliminating
lock contention
2025-08-11 23:31:25 +05:30
Shivam Mishra
ac3bce3932 fix: missing metrics and labels from label summary (#11718) 2025-06-12 17:58:56 +05:30
Shivam Mishra
35f06f30e7 feat: label reports overview (#11194)
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-06-11 14:35:46 +05:30
Muhsin Keloth
8565341682 fix: Prevent creating duplicate messages via Instagram echo events (#11535)
Fixes
https://linear.app/chatwoot/issue/CW-4383/copilot-resolution-message-creating-duplicate-conversations-due-to
and
https://linear.app/chatwoot/issue/CW-4287/instagram-channel-missing-existing-source-id-check

When the copilot/system resolves a conversation with a system resolve
message (with `lock_to_single_conversation` disabled), echo events were
creating new conversations instead of using existing ones. This occurred
because we were checking the echo_id in the new conversation rather than
in the resolved conversation. This PR fixes the issue by checking if the
message exists anywhere in the system instead of checking within a
particular conversation.
2025-05-22 11:02:28 +05:30
Sojan Jose
bc42aec68e chore: upgrade ruby version to 3.4.4 (#11524)
- Chore upgrade ruby version to 3.4.4 before we migrate to rails 7.2
over #11037
2025-05-21 19:40:07 +05:30
Vishnu Narayanan
df7401f71c fix: account email validation during signup (#11307)
- Refactor email validation logic to be a service
- Use the service for both email/pass signup and Google SSO
- fix account email validation during signup
- Use `blocked_domain` setting for both email/pass signup and Google
Sign In [`BLOCKED_DOMAIN` via GlobalConfig]
- add specs for `account_builder`
- add specs for the new service

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2025-05-20 20:45:39 -07:00
Muhsin Keloth
d827e66453 feat: Instagram Inbox using Instagram Business Login (#11054)
This PR introduces basic minimum version of **Instagram Business
Login**, making Instagram inbox setup more straightforward by removing
the Facebook Page dependency. This update enhances user experience and
aligns with Meta’s recommended best practices.

Fixes
https://linear.app/chatwoot/issue/CW-3728/instagram-login-how-to-implement-the-changes


## Why Introduce Instagram as a Separate Inbox?


Currently, our Instagram integration requires linking an Instagram
account to a Facebook Page, making setup complex. To simplify this
process, Instagram now offers **Instagram Business Login**, which allows
users to authenticate directly with their Instagram credentials.

The **Instagram API with Instagram Login** enables businesses and
creators to send and receive messages without needing a Facebook Page
connection. While an Instagram Business or Creator account is still
required, this approach provides a more straightforward integration
process.

| **Existing Approach (Facebook Login for Business)** | **New Approach
(Instagram Business Login)** |
| --- | --- |
| Requires linking Instagram to a Facebook Page | No Facebook Page
required |
| Users log in via Facebook credentials | Users log in via Instagram
credentials |
| Configuration is more complex | Simpler setup |

Meta recommends using **Instagram Business Login** as the preferred
authentication method due to its easier configuration and improved
developer experience.

---

## Implementation Plan

The core messaging functionality is already in place, but the transition
to **Instagram Business Login** requires adjustments.

### Changes & Considerations

- **API Adjustments**: The Instagram API uses `graph.instagram`, whereas
Koala (our existing library) interacts with `graph.facebook`. We may
need to modify API calls accordingly.
- **Three Main Modules**:
  1. **Instagram Business Login** – Handle authentication flow.
2. **Permissions & Features** – Ensure necessary API scopes are granted.
  3. **Webhooks** – Enable real-time message retrieval.

![CleanShot 2025-03-10 at 21 32
28@2x](https://github.com/user-attachments/assets/1b019001-8d16-4e59-aca2-ced81e98f538)


---

## Instagram Login Flow

1. User clicks **"Create Inbox"** for Instagram.
2. App redirects to the [Instagram Authorization
URL](https://developers.facebook.com/docs/instagram-platform/instagram-api-with-instagram-login/business-login#embed-the-business-login-url).
3. After authentication, Instagram returns an authorization code.
5. The app exchanges the code for a **long-lived token** (valid for 60
days).
6. Tokens are refreshed periodically to maintain access.
7. Once completed, the app creates an inbox and redirects to the
Chatwoot dashboard.

---

## How to Test the Instagram Inbox

1. Create a new app on [Meta's Developer
Portal](https://developers.facebook.com/apps/).
2. Select **Business** as the app type and configure it.
3. Add the Instagram product and connect a business account.
4. Copy Instagram app ID and Instagram app secret
5. Add the Instagram app ID and Instagram app secret to your app config
via `{Chatwoot installation
url}/super_admin/app_config?config=instagram`
6. Configure Webhooks:
   - Callback URL: `{your_chatwoot_url}/webhooks/instagram`
   - Verify Token: `INSTAGRAM_VERIFY_TOKEN`
- Subscribe to `messages`, `messaging_seen`, and `message_reactions`
events.
7. Set up **Instagram Business Login**:
   - Redirect URL: `{your_chatwoot_url}/instagram/callback`
8. Test inbox creation via the Chatwoot dashboard.


## Troubleshooting & Common Errors

### Insufficient Developer Role Error

- Ensure the Instagram user is added as a developer:
- **Meta Dashboard → App Roles → Roles → Add People → Enter Instagram
ID**

### API Access Deactivated

- Ensure the **Privacy Policy URL** is valid and correctly set.

### Invalid request: Request parameters are invalid: Invalid
redirect_uri

- Please configure the Frontend URL. The Frontend URL does not match the
authorization URL.
---


## To-Do List

- [x] Basic integration setup completed.  
- [x] Enable sending messages via [Messaging
API](https://developers.facebook.com/docs/instagram-platform/instagram-api-with-instagram-login/messaging-api).
- [x] Implement automatic webhook subscriptions on inbox creation.  
- [x] Handle **canceled authorization errors**.  
- [x] Handle all the errors
https://developers.facebook.com/docs/instagram-platform/instagram-graph-api/reference/error-codes
- [x] Dynamically fetch **account IDs** instead of hardcoding them.  
- [x] Prevent duplicate Instagram channel creation for the same account.
- [x] Use **Global Config** instead of environment variables.  
- [x] Explore **Human Agent feature** for message handling.  
- [x] Write and refine **test cases** for all scenarios.  
- [x] Implement **token refresh mechanism** (tokens expire after 60
days).
Fixes https://github.com/chatwoot/chatwoot/issues/10440

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2025-04-08 10:47:41 +05:30
Muhsin Keloth
41d6f9a200 chore: Add cache to improve widget performance (#11163)
- Add dynamic importing for routes.
- Added caching for `campaign`, `articles` and `inbox_members` API end
points.

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
2025-03-24 16:04:49 -07:00
Pranav
d355801555 fix: Do not allow sending messages if merged contact has a duplicate session (#11152)
In this PR https://github.com/chatwoot/chatwoot/pull/11139, if there is
an attempt to create a duplication session for the contact in the same
inbox, we will anonymize the old session.

This PR would prevent sending messages to the older sessions. The
support agents will have to create a new conversation to continue
messages with customer.
2025-03-21 18:04:46 -07:00
Pranav
b5deac468e fix: Fix duplicate contact inbox race condition (#11139)
This PR addresses a race condition in the contact inbox model caused by
duplicate `source_id` values linked to different contacts.

The issue typically occurs when an agent updates a contact’s email or
phone number or when two contacts are merged. In these scenarios, the
`source_id`, which is intended to uniquely identify the contact in a
session, may still be associated with the old contact inbox.

To solve this, we check if there’s already a ContactInbox with the same
source_id but linked to another contact. If we find one, we update that
old record by changing its source_id to a random value. This breaks the
wrong connection and prevents issues, while still keeping the old data
safe.

However, this is only a temporary fix. The main issue is with the way
the contact inbox model is designed. Right now, it’s being used to track
sessions, but that may not be necessary for non-live chat channels. In
the long run, we should consider redesigning this part of the system to
avoid such problems.
2025-03-20 18:24:28 -07:00
Sojan Jose
97235a4365 fix: Exclude notifications for blocked contacts, except mentions (#10659)
- Prevents notifications from being created for conversations or actions
involving blocked contacts.
- The exception is the conversation_mention notification type, which
will still be created when applicable.
2025-01-12 15:30:27 +05:30
Pranav
c52282307a feat(v4): Update team, agent summary builder to include resolution metrics (#10607)
Following https://github.com/chatwoot/chatwoot/pull/10604, this PR
introduces similar reporting features for Agents and Teams.

Updates in this PR:
- Added additional methods to the base class to avoid repetition.
- Improve reporting for Teams and Agents to include resolution count.
2024-12-20 19:16:56 +05:30
Pranav
4fd9bddb9d feat(v4): Add API to fetch aggregate reports for inboxes (#10604)
The Inbox Overview section is being updated to offer a more detailed
report, showing an overall view of the account grouped by inboxes. To
view detailed reports and access specific graphs for individual inboxes,
click on the inbox name to navigate to its dedicated report page.

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-12-19 14:47:19 -08:00
giquieu
472f6d9345 feat: Ability to lock the conversation to a single thread in API channels (#10329)
Added the possibility to mark as a single conversation in the API type
inbox. This allows the conversation builder to search for the last
conversation.
I thought about searching for the last conversation with created_at:
desc order, as is done in some channels... but I didn't change the way
the conversation is searched.

Fixes: #7726 

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-12-09 23:36:17 +08:00
Sojan Jose
8e2b329202 feat: Render instagram reels in Chatwoot (#9829)
- Previously we were ignoring the reels shared over Instagram messages.
This PR will render the reels with in Chatwoot.

followup : we need to render reels in a better interface so that it is
clearly denoted to the user that its an Instagram reel
2024-07-24 12:58:12 -07:00
Pranav
87d92f73d4 feat: Improve Report API performance (#9476)
- Re-write the methods for clarity
- Remove the dependency on the ReportHelper class.
- Remove n+1 queries in the average metric time series data.
2024-05-22 17:34:24 -07:00
Jaideep Guntupalli
e21d7552d3 feat: extending lock to single conversation to meta inbox (#9104)
This change introduces the ability to lock conversations to a single thread for Instagram and facebook messages within the Meta inbox, mirroring existing functionality in WhatsApp and SMS inboxes.

Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2024-04-03 21:33:23 +05:30
Shivam Kumar
1253264382 fix: Avoid duplicate invitation emails when adding an agent (#9131)
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
Co-authored-by: Pranav <pranav@chatwoot.com>
2024-03-26 18:58:45 -07:00
Sojan Jose
881d4bf644 feat: Add backend APIs for the bot metrics (#9031)
Co-authored-by: Pranav <pranav@chatwoot.com>
2024-03-01 08:20:20 -08:00
Shivam Mishra
8acc818f68 feat: allow bulk invite create via email (#8853)
* feat: add agent builder

* feat: use new agent builder

* refactor: validate limit

* test: agent limits

* feat: allow bulk create

* feat: allow bulk create

* refactor: rename current_user to inviter in AgentBuilder

* refactor: move limits tests to enterprise

* test: send correct params

* refactor: account builder returns both user and account_user

* chore: Revert "refactor: account builder returns both user and account_user"

This reverts commit 1419789871e8a3b8ff57af27fe53925b1486a839.

* feat: return user as is

* Update agent_builder.rb

- minor update

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-02-06 09:04:04 +05:30
Muhsin Keloth
818424259f chore: Get all notification API improvments (#8549)
Co-authored-by: Sojan Jose <sojan@chatwoot.com>
2024-01-17 09:02:18 +05:30
Shivam Mishra
8455186e9f feat: allow instagram reply_to [CW-2609] (#8248)
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>
2023-10-31 17:31:12 -07:00
Shivam Mishra
b9694a0818 feat: support reply to for Telegram (#8105)
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2023-10-20 13:14:20 +05:30
Vishnu Narayanan
415bb23c37 fix: Handle invalid metric in ReportsController (#8086)
Co-authored-by: Pranav Raj S <pranav@chatwoot.com>

Fixes CW-2643
2023-10-12 17:01:23 +05:30
Sojan Jose
6ab964b161 chore: fix flaky spec (#7686)
- fix flaky reporting specs
- fix flaky export spec
2023-08-25 00:11:41 -07:00
Pranav Raj S
d7566c453d chore: Take the count directly rather than grouping the conversations (#7535) 2023-07-19 12:12:30 -07:00
Shivam Mishra
2e79a32db7 fix: calculation for resolution count (#7293)
* fix: calculation for resolution count

* test: resolution count bug fix

- ensure enqueued jobs are run
- fix the dates check, conversations resolved today should show up in today

* feat: ensure conversations are resolved

* test: do not count extra events if the conversation is not resolved currently

* fix: typo
2023-06-14 13:50:10 +05:30
Sojan Jose
7ab7bac6bf chore: Enable the new Rubocop rules (#7122)
fixes: https://linear.app/chatwoot/issue/CW-1574/renable-the-disabled-rubocop-rules
2023-05-19 14:37:10 +05:30
Tejaswini Chile
65e91f6a6f fix: Duplicate instagram conversations for echo messages (#7078) 2023-05-15 16:25:15 +05:30
Tejaswini Chile
3c2d6faf68 feat: Instagram story replies will display the original story link (#6846) 2023-04-26 15:27:07 +05:30
Shivam Mishra
6b2736aa63 fix: inconsistency in report and summary for metric counts (#6817)
* feat: include timezone offset in summary calculation

* fix: exlcude end in date range

* test: explicit end of day

* fix: test for report builder

* fix: reports.spec.js

---------

Co-authored-by: Tejaswini Chile <tejaswini@chatwoot.com>
2023-04-20 12:55:04 +05:30
Jordan Brough
026e03c307 fix: Handle spaces in CC/BCC email lists (#6788)
When the CC field is generated in the UI, the email values are joined together
with ", " but when they are parsed, we currently split by just ",".

This causes an error on the backend and on the frontend.

It seems reasonable to update the code to allow whitespace in the input and to
split by `\s*,\s` and also to trim leading and trailing whitespace from the CC
list.

---------

Co-authored-by: Sojan <sojan@pepalo.com>
2023-04-18 19:18:23 +05:30
Tejaswini Chile
54b7c98795 fix: Warn Facebook error code 100-2018218 (#6632) 2023-03-09 13:51:10 +05:30