Incorrect character encoding when set custom attribute with special characters using the SOAP API #1407
Comments
|
Changing the SOAP.pm file fixes the error for me: Custom attribute are set with correct characters: getDetails: correct characters And correct characters on web interface: |
|
Hi @Bakaras , Just for information, is this problem specific to sympa_soap_client.pl? Or does it also occur in real SOAP applications? |
|
SOAP::Lite's UTF8 support seems to be buggy: https://rt.cpan.org/Public/Bug/Display.html?id=86649 |
|
@racke, I suspect that is a different issue. |
|
@ikedas, that is also possible. |
|
@Bakaras , additional question: Can you try with following patch (without your patch above)? diff --git a/src/bin/sympa_soap_client.pl.in b/src/bin/sympa_soap_client.pl.in
index 8e97c4d..4a9134a 100644
--- a/src/bin/sympa_soap_client.pl.in
+++ b/src/bin/sympa_soap_client.pl.in
@@ -26,6 +26,7 @@
use lib '--modulesdir--';
use strict;
use warnings;
+use Encode qw();
use Getopt::Long;
use HTTP::Cookies;
#use SOAP::Lite +trace;
@@ -170,6 +171,10 @@ sub play_soap_as_trusted {
printf
"calling authenticateRemoteAppAndRun( $trusted_application, $trusted_application_password, $proxy_vars,$service,$p)\n";
+ foreach my $param (@parameters) {
+ $param = Encode::decode_utf8($param);
+ }
+
my $reponse =
$soap->authenticateRemoteAppAndRun($trusted_application,
$trusted_application_password, $proxy_vars, $service, \@parameters);
@@ -264,6 +269,10 @@ sub play_soap {
@parameters = split /,/, $service_parameters;
}
+ foreach my $param (@parameters) {
+ $param = Encode::decode_utf8($param);
+ }
+
if ($service eq 'lists') {
printf "\n\nlists....\n";
$reponse = $soap->lists(); |
I have no real SOAP application. Without this change I have this output: |
It did not help |
That doesn't make sense to me. Why don't you use the Sympa CLI?
|
|
I think with Sympa CLI custom attributes are not imported (?) |
Is the result exactly the same as before applying the patch? Could you please show the actual result? |
|
Yes, the result is exactly the same. |
|
In addition: |
Oops, yes that is quite possible 😇 |
Interesting. How to reproduce it? Can you show the steps? BTW in your first example in above, umlaut is transformed to two characters but not three characters in your previous result. Is this difference caused by my patch? If it is, I might catch the tail of bug... |
Output: Sorry, have now eaten and drank some beer, left the real data as it is. |
It was probably the output after the second call. |
|
@Bakaras , sorry to interrupt your relaxation time:smile:. At the moment the fix you suggested looks promising, and I will prepare a PR based on it. |
|
@Bakaras , I found this workaround solves the problem. diff --git a/src/lib/Sympa/WWW/SOAP.pm b/src/lib/Sympa/WWW/SOAP.pm
index ba4caf9..6fc986b 100644
--- a/src/lib/Sympa/WWW/SOAP.pm
+++ b/src/lib/Sympa/WWW/SOAP.pm
@@ -1501,6 +1501,11 @@ sub setCustom {
#if(! defined $list->{'admin'}{'custom_attribute'}{$key} ) {
# return SOAP::Data->name('result')->type('boolean')->value(0);
#}
+
+ $log->syslog('info', 'key=%s:utf8=%s; value=%s:utf8=%s',
+ $key, Encode::is_utf8($key), $value, Encode::is_utf8($value));
+ $key = Encode::encode_utf8($key) if Encode::is_utf8($key);
+
if ($value eq '') {
undef $newcustom{$key};
} else {The log line added by this is This behavior seems due to bug (or buggy behavior) of SOAP::Lite. |
|
OK, with workaround, custom attribute "vorname" are set with correct characters. But after setCustom "nachname" is "vorname" broken again: |
|
In addition: |
|
@Bakaras , what is shown in the log? |
|
|
Thanks @Bakaras !
It is probably a bug in SOAP::Lite (but probably different from the one @racke referred to). What is the version of SOAP::Lite you use? I use 1.10. Anyways workaround would be as below: diff --git a/src/lib/Sympa/WWW/SOAP.pm b/src/lib/Sympa/WWW/SOAP.pm
index ba4caf9..6fc986b 100644
--- a/src/lib/Sympa/WWW/SOAP.pm
+++ b/src/lib/Sympa/WWW/SOAP.pm
@@ -1501,6 +1501,10 @@ sub setCustom {
#if(! defined $list->{'admin'}{'custom_attribute'}{$key} ) {
# return SOAP::Data->name('result')->type('boolean')->value(0);
#}
+
+ Encode::_utf8_off($key);
+ Encode::_utf8_off($value);
+
if ($value eq '') {
undef $newcustom{$key};
} else { |
|
|
@Bakaras , if you have no objection, I'll merge the PR above, and it will be included in the next release. |
|
OK, Jürgen is happy. And what about sympa_soap_client.pl? if (ref($r->result) =~ /^ARRAY/) {
#printf "R: $r->result\n";
@ret = @{$r->result};
+ }elsif (ref( $r->result) =~ /^HASH/) {
+ @ret = ( $r->result );
} elsif (ref $r->result) {
print "Pb " . ($r->result) . "\n";
return undef; |
Thanks!
I'd overlooked it. Thanks for the reminder! |
SympaSOAP: Result by setCustom with parameter including non-ASCII characters is broken (#1407)
…ympa-community#1541) This commit also reverts a redundant fix done at 39ddf25 (sympa-community#1407)
…ympa-community#1541) This commit also reverts a redundant fix done at 39ddf25 (sympa-community#1407)
…ympa-community#1541) This commit also reverts a redundant fix done at 39ddf25 (sympa-community#1407)
…ympa-community#1541) Revert a redundant fix which has been done at 39ddf25 (sympa-community#1407)
…ympa-community#1541) Revert a redundant fix which has been done at 39ddf25 (sympa-community#1407)
Version
6.2.60
[Note by admin: Patch by PR #1402 has to be applied]
Installation method
deb
Expected behavior
Custom attributes are set with correct characters.
Actual behavior
Custom attributes are set with incorrect characters.
Here with german umlauts: first name Jürgen.
Steps to reproduce
On web interface:
Additional information
The text was updated successfully, but these errors were encountered: