Template parsing problems in parameterizable data sources

[junior013]

Version

6.2.24

Installation method

deb (Ubuntu 18.04 LTS, bionic repo)

Expected behavior

Import members from predefined LDAP with filter as a parameter

Actual behavior

• If source_parameters defined -> Works
• If source_parameters left empty -> The unparsed template code goes in the LDAP query

Additional information

Template V1:
filter (&(cn=*)[% param.0 %])
log:
wwsympa[6381]: err main::#1606 > main::do_review#4842 > Sympa::List::on_the_fly_sync_include#6929 > Sympa::List::sync_include#6581 > Sympa::List::_load_list_members_from_include#5783 > Sympa::List::_include_users_ldap#5059 > Sympa::DatabaseDriver::LDAP::do_operation#204 Unable to perform LDAP operation: Bad filter
wwsympa[6381]: err main::#1606 > main::do_review#4842 > Sympa::List::on_the_fly_sync_include#6929 > Sympa::List::sync_include#6581 > Sympa::List::_load_list_members_from_include#5783 > Sympa::List::_include_users_ldap#5071 LDAP search (single level) failed: (89) Bad filter (searching on server X:389; suffix X; filter (&(cn=*)[% param.0 %]); attrs: mail)

Template V2:
[% SET filter = '' %]
[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = 'cn=*'; END %]
filter [% filter %]
log:
wwsympa[6381]: info Sympa::List::_load_include_admin_user_file() Bad line "[% SET filter = '' %]#012" in /etc/sympa/data_sources/ad_users.incl
wwsympa[6381]: info Sympa::List::_load_include_admin_user_file() Bad entry "[% SET filter = '' %]" for key "ALL", paragraph "include_ldap_query" in /etc/sympa/data_sources/ad_users.incl
wwsympa[6381]: info Sympa::List::_load_include_admin_user_file() Bad line "[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = 'cn=*'; END %]#012" in /etc/sympa/data_sources/ad_users.incl
wwsympa[6381]: info Sympa::List::_load_include_admin_user_file() Bad entry "[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = 'cn=*'; END %]" for key "include_ldap_query", paragraph "include_

Looks like template engine not processing directives, or the config include parser tries parsing the file before the template engine?

[ikedas]

Hi @junior013,

Could you please check if this patch will fix the problem you reported?

[junior013]

Hello @ikedas ,

Thank you! It works - with simple template filter (&(cn=*˙)[% param.0 %].
Complex (multiline) templates looks parsed too, but needs to omit empty lines to work as expected.

[ikedas]

@junior013, what is empty line? i couldn't figure out what happened.

[junior013]

@ikedas ,

Templates like
[% SET filter = '' %]
[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = 'cn=*'; END %]
filter [% filter %]

generates empty lines in first 2 line, end errors
Missing key "filter" in param "include_ldap_query" in /etc/sympa/data_sources/
Expecting a single line for "filter" parameter in /etc/sympa/data_sources/
Bad entry "filter cn=*" in /etc/sympa/data_sources/

But if I omit the newline-s in the template, Its OK:
[% SET filter = '' -%]
[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = '(cn=*)'; END -%]
filter [% filter %]

Maybe the parser could clean up any empty lines - if someone miss it.

[junior013]

@ikedas ,

I still get errors from the automatic regular refresh of the list members :(
list task_manager[31169]: info Sympa::List::_load_include_admin_user_file() Bad line "[% SET filter = '' -%]#012" in /etc/sympa/data_sources/ad_users.incl
list task_manager[31169]: info Sympa::List::_load_include_admin_user_file() Bad line "[% IF param.defined AND param.defined(0); filter = param.0; ELSE; filter = '(cn=*)'; END -%]#012" in /etc/sympa/data_sources/ad_users.incl
list task_manager[31169]: debug2 Sympa::List::_include_users_ldap() Searching on server X:389; suffix X; filter [% filter %]; attrs: mail
list task_manager[31169]: err main::#399 > main::execute#808 > main::cmd_process#878 > main::sync_include#2063 > Sympa::List::sync_include#6581 > Sympa::List::_load_list_members_from_include#5783 > Sympa::List::_include_users_ldap#5059 > Sympa::DatabaseDriver::LDAP::do_operation#204 Unable to perform LDAP operation: Bad filter

Is it another code then the manual sync?

[ikedas]

@junior013, I understand of empty lines. It’s unhandy things with Template Toolkit.

On errors with task_manager: have you restarted Sympa services?

[junior013]

On errors with task_manager: have you restarted Sympa services?

No, I'm not. Sorry! Now it seems OK.

(But, I commented another related problem in the code)

[ikedas]

@junior013,

the fix will be included in the next reease in December. If you have no more problem on this issue, please close it.

Thanks for reporting bug!

[junior013]

@ikedas

Thank you, it seems OK. I close it.