fix: Preserve multiple newlines with whitespace in text-based messaging channels (#13044)

## Description

Fixes an issue where multiple newlines with whitespace between them
(e.g., `\n \n \n`) were being collapsed to single newlines in text-based
messaging channels (Telegram, WhatsApp, Instagram, Facebook, Line, SMS).

The frontend was sending messages with spaces/tabs between newlines, and
the markdown renderer was treating these as paragraph content,
collapsing them during rendering.

### Changes:
1. Added whitespace normalization in `render_telegram_html`,
`render_whatsapp`, `render_instagram`, `render_line`, and
`render_plain_text` methods
2. Strips whitespace from whitespace-only lines before markdown
processing
3. Added comprehensive regression tests for all affected channels

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## How Has This Been Tested?

1. **Unit Tests**: Added 7 new specs testing multiple newlines with
whitespace between them for all text-based channels
2. **Manual Testing**: Verified with actual frontend payload containing
`\n \n \n` patterns
3. **Regression Testing**: All existing 63 specs pass

### Test Results:
-  All 63 markdown renderer specs pass (56 original + 7 new)
-  All 12 Telegram channel specs pass
-  All 27 WhatsApp + Instagram specs pass
-  Verified with real-world payload: 18 newlines preserved (previously
collapsed to 1)

### Test Command:
```bash
RAILS_ENV=test bundle exec rspec spec/services/messages/markdown_renderer_service_spec.rb
RAILS_ENV=test bundle exec rspec spec/models/channel/telegram_spec.rb
```

## Checklist:

- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
This commit is contained in:
Vinay Keerthi
2025-12-10 21:44:16 +05:30
committed by GitHub
parent 0d8e249fe4
commit 89d02e2c92
2 changed files with 53 additions and 6 deletions

View File

@@ -47,13 +47,19 @@ class Messages::MarkdownRendererService
end
def render_telegram_html
# Strip whitespace from whitespace-only lines to normalize newlines
normalized_content = @content.gsub(/^[ \t]+$/m, '')
content_with_preserved_newlines = preserve_multiple_newlines(normalized_content)
renderer = Messages::MarkdownRenderers::TelegramRenderer.new
doc = CommonMarker.render_doc(@content, [:STRIKETHROUGH_DOUBLE_TILDE], [:strikethrough])
renderer.render(doc).gsub(/\n+\z/, '')
doc = CommonMarker.render_doc(content_with_preserved_newlines, [:STRIKETHROUGH_DOUBLE_TILDE], [:strikethrough])
result = renderer.render(doc).gsub(/\n+\z/, '')
restore_multiple_newlines(result)
end
def render_whatsapp
content_with_preserved_newlines = preserve_multiple_newlines(@content)
# Strip whitespace from whitespace-only lines to normalize newlines
normalized_content = @content.gsub(/^[ \t]+$/m, '')
content_with_preserved_newlines = preserve_multiple_newlines(normalized_content)
renderer = Messages::MarkdownRenderers::WhatsAppRenderer.new
doc = CommonMarker.render_doc(content_with_preserved_newlines, [:DEFAULT, :STRIKETHROUGH_DOUBLE_TILDE])
result = renderer.render(doc).gsub(/\n+\z/, '')
@@ -61,7 +67,9 @@ class Messages::MarkdownRendererService
end
def render_instagram
content_with_preserved_newlines = preserve_multiple_newlines(@content)
# Strip whitespace from whitespace-only lines to normalize newlines
normalized_content = @content.gsub(/^[ \t]+$/m, '')
content_with_preserved_newlines = preserve_multiple_newlines(normalized_content)
renderer = Messages::MarkdownRenderers::InstagramRenderer.new
doc = CommonMarker.render_doc(content_with_preserved_newlines, [:DEFAULT, :STRIKETHROUGH_DOUBLE_TILDE])
result = renderer.render(doc).gsub(/\n+\z/, '')
@@ -69,7 +77,9 @@ class Messages::MarkdownRendererService
end
def render_line
content_with_preserved_newlines = preserve_multiple_newlines(@content)
# Strip whitespace from whitespace-only lines to normalize newlines
normalized_content = @content.gsub(/^[ \t]+$/m, '')
content_with_preserved_newlines = preserve_multiple_newlines(normalized_content)
renderer = Messages::MarkdownRenderers::LineRenderer.new
doc = CommonMarker.render_doc(content_with_preserved_newlines, [:DEFAULT, :STRIKETHROUGH_DOUBLE_TILDE])
result = renderer.render(doc).gsub(/\n+\z/, '')
@@ -77,7 +87,9 @@ class Messages::MarkdownRendererService
end
def render_plain_text
content_with_preserved_newlines = preserve_multiple_newlines(@content)
# Strip whitespace from whitespace-only lines to normalize newlines
normalized_content = @content.gsub(/^[ \t]+$/m, '')
content_with_preserved_newlines = preserve_multiple_newlines(normalized_content)
renderer = Messages::MarkdownRenderers::PlainTextRenderer.new
doc = CommonMarker.render_doc(content_with_preserved_newlines, [:DEFAULT, :STRIKETHROUGH_DOUBLE_TILDE])
result = renderer.render(doc).gsub(/\n+\z/, '')

View File

@@ -426,5 +426,40 @@ RSpec.describe Messages::MarkdownRendererService, type: :service do
expect(result).to eq(content)
end
end
# Shared test for all text-based channels that preserve multiple newlines
# This tests the real-world scenario where frontend sends newlines with whitespace between them
context 'when content has multiple newlines with whitespace between them' do
# This mimics what frontends often send: newlines with spaces/tabs between them
let(:content_with_whitespace_newlines) { "hello \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n\nhello wow" }
%w[
Channel::Telegram
Channel::Whatsapp
Channel::Instagram
Channel::FacebookPage
Channel::Line
Channel::Sms
].each do |channel_type|
context "when channel is #{channel_type}" do
it 'normalizes whitespace-only lines and preserves multiple newlines' do
result = described_class.new(content_with_whitespace_newlines, channel_type).render
# Should preserve most of the newlines (at least 10+)
# The exact count may vary slightly by renderer, but should be significantly more than 1-2
expect(result.scan("\n").count).to be >= 10
# Should not collapse everything to just 1-2 newlines
expect(result.scan("\n").count).to be > 5
end
end
end
context 'when channel is Channel::TwilioSms with WhatsApp' do
it 'normalizes whitespace-only lines and preserves multiple newlines' do
channel = instance_double(Channel::TwilioSms, whatsapp?: true)
result = described_class.new(content_with_whitespace_newlines, 'Channel::TwilioSms', channel).render
expect(result.scan("\n").count).to be >= 10
end
end
end
end
end