Commit Graph

182 Commits

Author SHA1 Message Date
Muhsin Keloth
bdcb080e40 feat: Handle instagram test service (#11244)
This PR will handle the Instagram test events. We are using the last
created Instagram channel as the test channel since we don't have any
other channels for testing purposes at the time of Meta approval.



https://github.com/user-attachments/assets/98302b7a-d72c-4950-9660-861a5e08d55f

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2025-04-11 19:11:29 +05:30
Muhsin Keloth
d827e66453 feat: Instagram Inbox using Instagram Business Login (#11054)
This PR introduces basic minimum version of **Instagram Business
Login**, making Instagram inbox setup more straightforward by removing
the Facebook Page dependency. This update enhances user experience and
aligns with Meta’s recommended best practices.

Fixes
https://linear.app/chatwoot/issue/CW-3728/instagram-login-how-to-implement-the-changes


## Why Introduce Instagram as a Separate Inbox?


Currently, our Instagram integration requires linking an Instagram
account to a Facebook Page, making setup complex. To simplify this
process, Instagram now offers **Instagram Business Login**, which allows
users to authenticate directly with their Instagram credentials.

The **Instagram API with Instagram Login** enables businesses and
creators to send and receive messages without needing a Facebook Page
connection. While an Instagram Business or Creator account is still
required, this approach provides a more straightforward integration
process.

| **Existing Approach (Facebook Login for Business)** | **New Approach
(Instagram Business Login)** |
| --- | --- |
| Requires linking Instagram to a Facebook Page | No Facebook Page
required |
| Users log in via Facebook credentials | Users log in via Instagram
credentials |
| Configuration is more complex | Simpler setup |

Meta recommends using **Instagram Business Login** as the preferred
authentication method due to its easier configuration and improved
developer experience.

---

## Implementation Plan

The core messaging functionality is already in place, but the transition
to **Instagram Business Login** requires adjustments.

### Changes & Considerations

- **API Adjustments**: The Instagram API uses `graph.instagram`, whereas
Koala (our existing library) interacts with `graph.facebook`. We may
need to modify API calls accordingly.
- **Three Main Modules**:
  1. **Instagram Business Login** – Handle authentication flow.
2. **Permissions & Features** – Ensure necessary API scopes are granted.
  3. **Webhooks** – Enable real-time message retrieval.

![CleanShot 2025-03-10 at 21 32
28@2x](https://github.com/user-attachments/assets/1b019001-8d16-4e59-aca2-ced81e98f538)


---

## Instagram Login Flow

1. User clicks **"Create Inbox"** for Instagram.
2. App redirects to the [Instagram Authorization
URL](https://developers.facebook.com/docs/instagram-platform/instagram-api-with-instagram-login/business-login#embed-the-business-login-url).
3. After authentication, Instagram returns an authorization code.
5. The app exchanges the code for a **long-lived token** (valid for 60
days).
6. Tokens are refreshed periodically to maintain access.
7. Once completed, the app creates an inbox and redirects to the
Chatwoot dashboard.

---

## How to Test the Instagram Inbox

1. Create a new app on [Meta's Developer
Portal](https://developers.facebook.com/apps/).
2. Select **Business** as the app type and configure it.
3. Add the Instagram product and connect a business account.
4. Copy Instagram app ID and Instagram app secret
5. Add the Instagram app ID and Instagram app secret to your app config
via `{Chatwoot installation
url}/super_admin/app_config?config=instagram`
6. Configure Webhooks:
   - Callback URL: `{your_chatwoot_url}/webhooks/instagram`
   - Verify Token: `INSTAGRAM_VERIFY_TOKEN`
- Subscribe to `messages`, `messaging_seen`, and `message_reactions`
events.
7. Set up **Instagram Business Login**:
   - Redirect URL: `{your_chatwoot_url}/instagram/callback`
8. Test inbox creation via the Chatwoot dashboard.


## Troubleshooting & Common Errors

### Insufficient Developer Role Error

- Ensure the Instagram user is added as a developer:
- **Meta Dashboard → App Roles → Roles → Add People → Enter Instagram
ID**

### API Access Deactivated

- Ensure the **Privacy Policy URL** is valid and correctly set.

### Invalid request: Request parameters are invalid: Invalid
redirect_uri

- Please configure the Frontend URL. The Frontend URL does not match the
authorization URL.
---


## To-Do List

- [x] Basic integration setup completed.  
- [x] Enable sending messages via [Messaging
API](https://developers.facebook.com/docs/instagram-platform/instagram-api-with-instagram-login/messaging-api).
- [x] Implement automatic webhook subscriptions on inbox creation.  
- [x] Handle **canceled authorization errors**.  
- [x] Handle all the errors
https://developers.facebook.com/docs/instagram-platform/instagram-graph-api/reference/error-codes
- [x] Dynamically fetch **account IDs** instead of hardcoding them.  
- [x] Prevent duplicate Instagram channel creation for the same account.
- [x] Use **Global Config** instead of environment variables.  
- [x] Explore **Human Agent feature** for message handling.  
- [x] Write and refine **test cases** for all scenarios.  
- [x] Implement **token refresh mechanism** (tokens expire after 60
days).
Fixes https://github.com/chatwoot/chatwoot/issues/10440

---------

Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2025-04-08 10:47:41 +05:30
Pranjal Kushwaha
0dc2af3c78 feat: Ability to delete account for administrators (#1874)
## Description

Add account delete option in the user account settings.

Fixes #1555 

## Type of change

- [ ] New feature (non-breaking change which adds functionality)


![image](https://user-images.githubusercontent.com/40784971/110349673-edcc5200-8058-11eb-8ded-a31d15aa0759.png)

![image](https://user-images.githubusercontent.com/40784971/110349778-0c324d80-8059-11eb-9291-abfbffedde5e.png)


## Checklist:

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented on my code, particularly in hard-to-understand
areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] Any dependent changes have been merged and published in downstream
modules

---------

Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Sojan Jose <sojan.official@gmail.com>
Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2025-04-03 10:41:39 +05:30
Vishnu Narayanan
0175714d65 feat: add job to remove stale contacts and contact_inboxes (#11186)
- Add a job to remove stale contacts and contact_inboxes across all accounts

Stale anonymous contact is defined as 
- have no identification (email, phone_number, and identifier are NULL)
- have no conversations
- are older than 30 days

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2025-03-28 12:18:39 +05:30
Sojan Jose
991b108a35 feat: discard Twilio events when Body parameter is not present (#11096)
- Discard Twilio events when body parameter is not present.

---------

Co-authored-by: Pranav <pranavrajs@gmail.com>
2025-03-17 12:47:11 +05:30
Sojan Jose
586dc800bb chore: Move Twilio event processing to background job (#11094)
- Twilio events were being processed synchronously, leading to slow API
responses.
- This change moves Twilio event processing to a background job to
improve performance and align with how other events (e.g., WhatsApp) are
handled.

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-03-15 13:51:08 -07:00
Sojan Jose
29158e32fe chore: Logger for non-existent WhatsApp channels (#11064)
- Add a warning logger for cases where we are getting webhook events for
inactive numbers.
- Add config to discard events for inactive numbers so that the meta
will stop sending events

---------

Co-authored-by: Pranav <pranav@chatwoot.com>
2025-03-12 15:50:38 -07:00
Sojan Jose
d0631e99a5 chore: Add warning logs when Chatwoot receives events for inactive channels (#11066) 2025-03-12 08:16:47 -07:00
Pranav
d017156f32 fix: Disable syncing IMAP if the account is suspended (#11031)
This PR disables the IMAP syncing if the account is suspended.
2025-03-05 17:10:24 -08:00
Pranav
4b12a8a51e chore: Add more conversation events for reload (#10877)
Followup PR for https://github.com/chatwoot/chatwoot/pull/10876. This PR
just adds all the events related to conversation update to be reloaded
before sending it to the UI.
2025-02-11 00:33:45 -08:00
Pranav
3c78d25306 chore: Reload conversation data in ActionCableBroadcastJob before sending (#10876)
During high-traffic periods, events may appear out of order, causing the
conversation job to queue outdated data, which can lead to issues in the
UI. This update ensures that only the latest available data is sent to the UI.

The conversation object is refreshed before sending it to the UI.
2025-02-10 23:16:15 -08:00
Pranav
d070743383 feat(ee): Add Captain features (#10665)
Migration Guide: https://chwt.app/v4/migration

This PR imports all the work related to Captain into the EE codebase. Captain represents the AI-based features in Chatwoot and includes the following key components:

- Assistant: An assistant has a persona, the product it would be trained on. At the moment, the data at which it is trained is from websites. Future integrations on Notion documents, PDF etc. This PR enables connecting an assistant to an inbox. The assistant would run the conversation every time before transferring it to an agent.
- Copilot for Agents: When an agent is supporting a customer, we will be able to offer additional help to lookup some data or fetch information from integrations etc via copilot.
- Conversation FAQ generator: When a conversation is resolved, the Captain integration would identify questions which were not in the knowledge base.
- CRM memory: Learns from the conversations and identifies important information about the contact.

---------

Co-authored-by: Vishnu Narayanan <vishnu@chatwoot.com>
Co-authored-by: Sojan <sojan@pepalo.com>
Co-authored-by: iamsivin <iamsivin@gmail.com>
Co-authored-by: Sivin Varghese <64252451+iamsivin@users.noreply.github.com>
2025-01-14 16:15:47 -08:00
Vishnu Narayanan
9a405d65ba fix: RemoveStaleRedisKeys service (#10562)
Fixes issues with RemoveStaleRedisKeys service

Co-authored-by: Sojan <sojan@pepalo.com>
2024-12-11 03:38:25 +08:00
Vishnu Narayanan
e95680e800 feat: remove stale ONLINE_PRESENCE contact keys in redis (#9558)
50% of Redis memory size comes from ONLINE_PRESENCE keys. This PR adds a
periodic job to remove stale keys from all accounts.
2024-12-02 15:47:54 +05:30
Sojan Jose
7a3303e841 fix: Undefined method `encode' for nil for avatar from url job (#10450)
Invalid urls supplied to the job was causing sentry issues. The issue primarily occurs when the download file.original_filename comes out as empty

fixes: https://github.com/chatwoot/chatwoot/issues/10449
2024-11-19 14:24:01 -08:00
Sojan Jose
0331815cc5 feat: Integration with Captain (alpha) (#9834)
- Integration with captain (alpha)

Co-authored-by: Pranav <pranavrajs@gmail.com>
2024-07-25 14:24:04 -07:00
Shivam Mishra
650fee58a6 feat: add Google Email fetch and OAuth token refresh service (#9603)
This PR adds the following changes

1. Add `Imap::GoogleFetchEmailService` and
`Google::RefreshOauthTokenService`. The
`Google::RefreshOauthTokenService` uses
`OmniAuth::Strategies::GoogleOauth2` which is already added as a packge
2. Update `Inboxes::FetchImapEmailsJob` to handle Google inboxes
3. Add SMTP settings for Google in `ConversationReplyMailerHelper` to
allow sending emails


## Preview

#### Incoming emails

![CleanShot 2024-06-06 at 17 17
22@2x](https://github.com/chatwoot/chatwoot/assets/18097732/9d7d70d1-68e3-4c16-b1ca-e5a2e6f890e8)

#### Outgoing email

![CleanShot 2024-06-06 at 17 18
05@2x](https://github.com/chatwoot/chatwoot/assets/18097732/1b4abf0e-e311-493e-bdc8-386886afbb25)

---------

Co-authored-by: Muhsin Keloth <muhsinkeramam@gmail.com>
2024-06-11 14:22:03 +05:30
Muhsin Keloth
d649bd2dfd feat: Add support for bulk snooze until (#9360) 2024-05-08 08:55:31 +05:30
Shivam Mishra
2ef767d60f feat: exporting contacts takes the filters into account (#9347)
- This PR allows contacts to be exported using the current filter in CRM view

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-05-07 20:18:36 -07:00
Pranav
de044e29f0 chore: Rescue Oauth2::Error and mark the channel as inactive (#9358)
Co-authored-by: Sojan <sojan@pepalo.com>
2024-05-07 12:03:38 -07:00
Pranav
2c3e7d6554 chore: Temporarily disable the disconnect logic for inboxes (#9321)
* chore: Temporarily disable the disconnect logic for inboxes

* Remove unused spec
2024-04-30 07:49:37 -07:00
Pranav
77db0d0701 feat: Add configurable interval for IMAP sync (#9302) 2024-04-25 18:58:20 -07:00
Sojan Jose
fd832d7593 chore: Add verbose errors for IMAP email channels (#9116) 2024-03-15 20:53:23 -07:00
Sojan Jose
41e269e873 feat(ee): Add reporting events for bots (#9027)
Added a new event conversation_bot_resolved and added a job to auto resolve the bot conversations if there was no activity for the last 1 hour.
2024-02-27 14:53:28 -08:00
Muhsin Keloth
e6cf8c39b7 feat: Update last_snoozed_at after the un-snooze notification (#8943) 2024-02-20 18:55:39 +05:30
Liam
5d9fb55370 feat: Export contact improvements (#8895)
This pull request enhances the export contacts feature by adding a confirmation step before exporting. Previously, clicking the export button would trigger the export action without confirmation.

Additionally, it ensures that only the intended recipient receives the export email, addressing the previous behaviour where all administrators received it.

Fixes: #8504

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-02-20 17:11:03 +05:30
Muhsin Keloth
fe744abe26 fix: Mark notification as unread after the un-snooze (#8917) 2024-02-16 17:30:54 +05:30
Pranav Raj S
eb379e1849 fix(refactor): Cleanup the specs and the logic for FetchIMAP job (#8766) 2024-02-10 14:03:50 -08:00
Vishnu Narayanan
c1d07a5471 feat: Add sidekiq jobs to monitor applied SLAs (#8828)
Fixes: https://linear.app/chatwoot/issue/CW-2983/sidekiq-jobservice-to-monitor-sla-breach

Co-authored-by: Sojan <sojan@pepalo.com>
2024-02-07 21:44:56 +04:00
Muhsin Keloth
39e27d2a23 feat: Delete all/read notifications (#8844) 2024-02-05 13:33:05 +05:30
Pranav Raj S
33e98bf61a fix: Use BODY.PEEK[HEADER] to avoid parsing issues with mail providers (#8833)
Co-authored-by: Sojan <sojan@pepalo.com>
2024-02-01 17:24:59 -08:00
Sojan Jose
d10525a714 fix: Template sync issue for older Whatsapp channels (#8830)
- This PR introduces a modification to the channel fetching logic, ensuring that channels with older message_template_last_updated timestamps are prioritized during synchronization.
2024-01-31 18:38:01 +04:00
Sojan Jose
390cd756e8 chore: Reorganize the installation config settings (#8794)
- Reorganizing installation config settings to move more configurations into UI from environment variables
- Changes to installation config to support premium plans in the enterprise edition
- Fixes the broken premium indicator in account/show and accounts/edit page
2024-01-31 16:48:42 +04:00
Shivam Mishra
3760f206e8 fix: mutex timeout and error handling (#8770)
Fixes the follow cases 
- The ensure block released the lock even on LockAcquisitionError
- Custom timeout was not allowed

This also refactored the with_lock method, now the key has to be constructed in the parent function itself

Co-authored-by: Sojan Jose <sojan@pepalo.com>
2024-01-24 14:18:21 +04:00
Surabhi Suman
4b40c61201 feat: Support Regex validation for custom attributes (#7856)
This allows a user to add/update a custom regex and a cue while defining custom attributes(Only applicable for type- text).
While adding/editing custom attributes, the values are validated against the attribute definition regex, and if it is incorrect, a cue message or default error message is shown and restricts invalid values from being saved.

Fixes: #6866
2024-01-23 18:01:57 +04:00
Sojan Jose
bc04d81a5a fix: Handle Net::IMAP::InvalidResponseError Exception bad response type "ESMTP" (#8755) 2024-01-22 15:32:44 +05:30
Pranav Raj S
29106ebc4b fix: raise_lock_acquisition_error if the job cannot set the lock (#8744)
Consider a scenario where two jobs are concurrently accessing a job with a lock mechanism. If the second job accesses lock_manager.locked? before the first job called lock_manager.lock(lock_key), it would return a false positive. At this point, both jobs can be executed freely.

To address this issue, the following change ensures that only the current thread that has set the lock can execute. Otherwise, a LockAcquisitionError will be thrown.

Co-authored-by: Sojan Jose <sojan@pepalo.com>
Co-authored-by: Shivam Mishra <scm.mymail@gmail.com>
2024-01-19 14:31:41 +04:00
Pranav Raj S
a579684b34 fix: Update the message-id check to use messageId parsing from mail gem (#8743)
- Update the message ID parsing logic to use the mail gem
- Update the code to improve readability
2024-01-19 14:29:04 +04:00
Muhsin Keloth
0ac015ce7a feat: Add a job to clear notifications that were created before 1 month (#8732) 2024-01-19 13:03:23 +05:30
Muhsin Keloth
19474e0074 chore: Improve active job error logs for deserialization error (#8742)
- Improve active job error logs for deserialization error 

Co-authored-by: Sojan <sojan@pepalo.com>
2024-01-18 19:27:18 +04:00
Pranav Raj S
1f4d860d9d fix: Use channel.inbox instead of inbox (#8734) 2024-01-18 00:46:25 -08:00
Pranav Raj S
fdbb3bf4b1 fix: Optimize email fetching logic to fix bandwidth exceeded issues (#8730)
The Inboxes::FetchImapEmailsJob is designed to fetch the entire email object and check if its message id is already in the database. However, this process is resource-intensive and time-consuming, as fetching the full email object takes a significant amount of time.

On average, fetching 100 emails can take approximately 3-4 minutes to complete depending on the IMAP server. Since we are not using server flags to identify which emails have already been fetched (to avoid compatibility issues with flags in different email services), we have to fetch all previously available emails. Currently we fetch all the messages that were created from yesterday. This becomes problematic with services like Gmail, which throttle requests based on bandwidth usage.

To address this issue, I have updated the logic as follows:

Fetch the sequence IDs of all the mails in the "Inbox" that were created from yesterday.
Use the FETCH command to fetch only the message-ids using BODY.PEEK[HEADER.FIELDS (MESSAGE-ID)] with the sequence IDs we got in the previous step. This is a faster operation with lower bandwidth usage, as it only returns the sequence ID and message ID.
Check if the message IDs are already present in the database for the selected inbox.
If not present, fetch the entire email object using the FETCH command and create the message.
If the message ID is already present, ignore it and move on to the next message-id.
I have also addressed the issue of duplicate emails appearing in conversations when two fetch email jobs occur simultaneously. I have added a lock for the channel to ensure that the job gracefully exits without waiting for the current job to complete.

Fixes #7247
Fixes #6082
Fixes #8314

Co-authored-by: Sojan <sojan@pepalo.com>
2024-01-18 11:45:16 +04:00
Muhsin Keloth
c899cc825d fix: Handle Contact import MalformedCSVError (#8706) 2024-01-18 13:05:58 +05:30
Muhsin Keloth
818424259f chore: Get all notification API improvments (#8549)
Co-authored-by: Sojan Jose <sojan@chatwoot.com>
2024-01-17 09:02:18 +05:30
Shivam Mishra
10076c6a3e feat: set lock timeout to 1 second (#8661) 2024-01-12 08:28:07 +05:30
Muhsin Keloth
68062216e4 feat: Add a job to reopen snoozed notifications (#8545) 2024-01-05 15:13:22 +05:30
Pranav Raj S
00eb5b152a chore(migration): Re-run database migration job for caching. (#8581)
A support request that came to Chatwoot Cloud revealed that the job was timed prematurely. The default timeout for Sidekiq queues was set as 25 minutes in sidekiq.yml file. The cache was not created properly for the accounts with more than 100k conversations.

This change removes the cache logic in the previous migration and creates a new migration with a new job which processes conversations in batch.
2023-12-18 17:29:02 -08:00
Sojan Jose
0b2f539ad0 chore: Update self-hosted billing redirect (#8580) 2023-12-18 16:34:47 -08:00
Pranav Raj S
890515edfd feat(perf): Cache labels on the conversation model (#8527) 2023-12-11 18:27:55 -08:00
Muhsin Keloth
80ff5e2d0a chore: Change the RemoveMessageNotifications job implementation. (#8521) 2023-12-11 08:27:30 +05:30