fix: stream attachment handling in workers (#12870)

We’ve been watching Sidekiq workers climb from ~600 MB at boot to
1.4–1.5 GB after an hour whenever attachment-heavy jobs run. This PR is
an experiment to curb that growth by streaming attachments instead of
loading the whole blob into Ruby: reply-mailer inline attachments,
Telegram uploads, and audio transcriptions now read/write in chunks. If
this keeps RSS stable in production we’ll keep it; otherwise we’ll roll
it back and keep digging
This commit is contained in:
Sojan Jose
2025-12-05 13:02:53 -08:00
committed by GitHub
parent a971ff00f8
commit cc86b8c7f1
12 changed files with 203 additions and 74 deletions

View File

@@ -29,12 +29,16 @@ class Captain::Llm::PdfProcessingService < Llm::LegacyBaseOpenAiService
end
end
def with_tempfile(&)
def with_tempfile
Tempfile.create(['pdf_upload', '.pdf'], binmode: true) do |temp_file|
temp_file.write(document.pdf_file.download)
temp_file.close
document.pdf_file.blob.open do |blob_file|
IO.copy_stream(blob_file, temp_file)
end
File.open(temp_file.path, 'rb', &)
temp_file.flush
temp_file.rewind
yield temp_file
end
end
end