KEMBAR78
Unicode 16 by JMazurkiewicz · Pull Request #5571 · microsoft/STL · GitHub
Skip to content

Unicode 16 #5571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Oct 17, 2025
Merged

Unicode 16 #5571

merged 12 commits into from
Oct 17, 2025
+1,450 −552

Conversation

@JMazurkiewicz
Copy link
Contributor

  • Implement generation of the Indic_Conjunct_Break table in the unicode_properties_data_gen.py script. We need this table to implement the new segmentation rule.
  • Update __msvc_format_ucd_tables.hpp.
  • Implement GB9c segmentation rule (added in Unicode 15.1) in _Grapheme_break_property_iterator.
  • Update tests.

Previous PR: #3556

@JMazurkiewicz JMazurkiewicz requested a review from a team as a code owner June 5, 2025 21:43
@github-project-automation github-project-automation bot moved this to Initial Review in STL Code Reviews Jun 5, 2025
@StephanTLavavej StephanTLavavej added enhancement Something can be improved format C++20/23 format labels Jun 5, 2025
@StephanTLavavej StephanTLavavej self-assigned this Jun 5, 2025
@frederick-vs-ja
Copy link
Contributor

frederick-vs-ja commented Jun 21, 2025 •
edited
Loading

FWIW, this is also related to CWG-2843 which is recently accepted. No change requested - Unicode 16 is definitely OK as the resolution requires Unicode 15.1 as the minimum version.

@StephanTLavavej
Merge branch 'main' into unicode16
b5c3e2b
@StephanTLavavej
Add break;.
f5166a7
@StephanTLavavej
Fix ABI, part 1: _Grapheme_break_property_iterator => `_Grapheme_br…
16ae0e6 
…eak_property_iterator2`

This is gaining a new data member `_GB9c_regex _GB9c_rx;`.
@StephanTLavavej
Fix ABI, part 2: _Measure_string_prefix_iterator_utf => `_Measure_s…
899de2e 
…tring_prefix_iterator_utf2`

This contains `_Grapheme_break_property_iterator2<_CharT> _WrappedIter;`.

This is the final part. The alias `_Measure_string_prefix_iterator` can expand to this type,
but aliases don't need to be renamed. The only use of that alias is within the function
`_Measure_string_prefix()`, so renaming these two classes is sufficient to preserve ABI.
@StephanTLavavej
Comment: "update it" => "update them", also rewrap.
f7856a7
@StephanTLavavej StephanTLavavej mentioned this pull request Oct 16, 2025
Open
StephanTLavavej
StephanTLavavej reviewed Oct 16, 2025
View reviewed changes
@StephanTLavavej
Copy link
Member

Thanks! 😻 And apologies for taking over 4 months to review this.

I pushed a conflict-free (albeit large) merge with main, a couple of nitpick commits, and a couple of class renames to preserve ABI.

I verified that the product and test code are exactly generated by the scripts (after clang-formatting).

@StephanTLavavej StephanTLavavej removed their assignment Oct 16, 2025
@StephanTLavavej

This comment was marked as resolved.

Sign in to view
@github-project-automation github-project-automation bot moved this from Initial Review to Done in STL Code Reviews Oct 16, 2025
@github-project-automation github-project-automation bot moved this from Done to Initial Review in STL Code Reviews Oct 16, 2025
@StephanTLavavej StephanTLavavej moved this from Initial Review to Ready To Merge in STL Code Reviews Oct 16, 2025
@StephanTLavavej
Copy link
Member

I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed.

@StephanTLavavej StephanTLavavej moved this from Ready To Merge to Merging in STL Code Reviews Oct 16, 2025
@AraHaan
Copy link

AraHaan commented Oct 17, 2025

Can we get a PR for Unicode 17 soon as well please?

@cpplearner
Copy link
Contributor

cpplearner commented Oct 17, 2025 •
edited
Loading

Can we get a PR for Unicode 17 soon as well please?

You can do it yourself. Just run the scripts under tools/unicode_properties_parse (download first, then the other two scripts), and replace the affected contents (entire stl/inc/__msvc_format_ucd_tables.hpp, part of tests/std/tests/P0645R10_text_formatting_grapheme_clusterization/test.cpp) with corresponding tool output.

@StephanTLavavej StephanTLavavej merged commit 7f05724 into microsoft:main Oct 17, 2025
39 checks passed
@github-project-automation github-project-automation bot moved this from Merging to Done in STL Code Reviews Oct 17, 2025
@StephanTLavavej
Copy link
Member

Thanks for this highly nontrivial PR! 🐱 🐈 🐈‍⬛

@JMazurkiewicz JMazurkiewicz deleted the unicode16 branch October 17, 2025 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@StephanTLavavej StephanTLavavej StephanTLavavej approved these changes

Assignees

No one assigned

Labels

enhancement Something can be improved format C++20/23 format

Projects

STL Code Reviews
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@JMazurkiewicz @frederick-vs-ja @StephanTLavavej @AraHaan @cpplearner