Fix/cjk unicode lookup

[Kanahiro]

Launch Checklist

• Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!

• Briefly describe the changes in this PR.

• Link to related issues.

• Include before/after visuals or gifs if this PR includes visual changes.

• Write tests for all new functionality.

• Add an entry to CHANGELOG.md under the ## main section.

• Issue: [label] Full-width hyphen in Japanese ー is accidently disappeared #5420

before

• あーい、う。 becomes あいう. some symbols are ignored.

after

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.95%. Comparing base (bedcb86) to head (adf4218).
Report is 8 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5421   +/-   ##
=======================================
  Coverage   91.94%   91.95%           
=======================================
  Files         282      282           
  Lines       39039    39050   +11     
  Branches     6849     6860   +11     
=======================================
+ Hits        35896    35907   +11     
  Misses       3016     3016           
  Partials      127      127           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[HarelM]

Thanks for taking the time to solve this!
Please add a render test to make sure this behavior is tested and kept in the future.
Also looks like a render test failed due to the changes made, please are that you take care of it as well.

[Kanahiro]

@1ec5 @HarelM
Thanks for reviewing!
I improved mycodes and added tests. Please check it.

[Kanahiro]

Tests / Render tests (windows-latest, 0) (pull_request) faled but rendered image of label could have difference between Ubuntu and Windows. I'm not sure how can I clear this. Please advice me if anyone have knowledge.

I understand I should make expected image as expected-ubuntu.png.
I wonder if we have to make on all OS. I have all machines but it may be not easy for many developers..

[Kanahiro]

I can't run $env:UPDATE=$true; npm run test-render -- text-local-ideographs/cjk-symbols in Windows.
Could anyone run and generate expected-windows.png?

I followed this docs but it doesn't work for me.
https://github.com/maplibre/maplibre-gl-js/tree/main/test/integration#updating-results-of-render-test-results

[HarelM]

You can download the render test html report from the failing CI run, open it in the browser and right click to save the image and check it in.
I guess this option should be mentioned in the readme as well...

[HarelM]

You don't have to create all the image, just those that are different between OSs.
In this case, because it is using system fonts you get different results, this is not true for most cases but we have the relevant infrastructure to solve this.

[Kanahiro]

@HarelM Thanks for advice! fixed