Bulk.pl don’t deliver mails if failed to personalize

[ldidry]

Version

Sympa 6.2.62

Installation method

Sources

Expected behavior

I want bulk.pl send messages even if there is a problem with the personalization.

Actual behavior

I have a message_global_footer file in /home/sympa/etc/mail_tt2, which content is added to all ML messages.

Some users send mails with at least one part declared as UTF-8 encoded but in fact Latin-1 encoded (please, don’t ask how they do it… it’s beyond my understanding).

Bulk fails to decode the part, then fails to add the footer and then don’t deliver the message to subscribers. Here’s a log :

May 17 14:03:42 rod3 bulk[9933]: err main::#160 > Sympa::Spindle::spin#83 > Sympa::Spindle::ProcessOutgoing::_twist#285 > Sympa::Message::personalize#1410 > Sympa::Message::_merge_msg#1464 > Sympa::Message::_merge_msg#1519 Cannot decode by charset "UTF-8"
May 17 14:03:42 rod3 bulk[9933]: err main::#160 > Sympa::Spindle::spin#83 > Sympa::Spindle::ProcessOutgoing::_twist#285 > Sympa::Message::personalize#1410 > Sympa::Message::_merge_msg#1465 Failed to personalize message part
May 17 14:03:42 rod3 bulk[9933]: err main::#160 > Sympa::Spindle::spin#83 > Sympa::Spindle::ProcessOutgoing::_twist#286 Erreur d appel personalize()
May 17 14:03:42 rod3 bulk[9933]: notice Sympa::Spindle::ProcessTemplate::_twist() Processing Sympa::Message::Template <[email protected],7454/shelved:dkim_sign>; [email protected]; [email protected]; recipients=ARRAY; [email protected]; template=listmaster_notification; type=bulk_failed

Additional information

I wrote a patch which avoid the total delivering failure:

diff --git a/src/lib/Sympa/Message.pm b/src/lib/Sympa/Message.pm
index 624ca5157..57dfcfb0f 100644
--- a/src/lib/Sympa/Message.pm
+++ b/src/lib/Sympa/Message.pm
@@ -1513,7 +1513,7 @@ sub _merge_msg {
         }
         unless ($in_cset->decoder) {
             $log->syslog('err', 'Unknown charset "%s"', $charset);
-            return undef;
+            return $entity;
         }
         $in_cset->encoder($in_cset);    # no charset conversion

@@ -1521,7 +1521,7 @@ sub _merge_msg {
         eval { $utf8_body = Encode::encode_utf8($in_cset->decode($body, 1)); };
         if ($EVAL_ERROR) {
             $log->syslog('err', 'Cannot decode by charset "%s"', $charset);
-            return undef;
+            return $entity;
         }

         ## PARSAGE ##

It’s clearly not perfect as the delivered mail have encoding problems (you can have tofu in it) but it’s better than not having mail at all 🤷

[ikedas]

Hi @ldidry ,
I don't know why delivery was aborted when personalization has failed.
Anyways I fixed it according to your suggestion. Thank you!

[ldidry]

Thank you!