Commit graph

27 commits

Author SHA1 Message Date
Stephen Dennis
14541d5335 Backport Unicode fixes: pairs DFA, collation NFC tiebreaker, tool updates
Port three fixes from libutf/master:

1. Fix pairs DFA generator: always resolve undefined transitions
   before optimization. The old code only called SetUndefinedStates
   when -d was passed, causing NFC over-composition and DUCET
   contraction false-matches.

2. Regenerate NFC compose tables (129→1010 states) and DUCET
   contraction tables (27→294 states) with the corrected generator.

3. NFC tiebreaker in collation: normalize to NFC before binary
   comparison so canonically equivalent strings compare equal per UCA.

Also updates all four table generator tools (classify, integers,
strings, pairs) to support -o/-i flags for output file control,
and adds LIBMUX_API compat define to Makefile.in preamble.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:44:40 -06:00
Stephen Dennis
dbd7cb4267 Use nullptr instead of NULL. 2025-03-30 09:10:10 -06:00
Stephen Dennis
b92a76c0e9 Small clean up of smutil.cpp. 2025-03-30 07:27:12 -06:00
Stephen Dennis
2a9061df5b COPY phase can be -128 2025-03-25 07:23:14 -06:00
Stephen Dennis
429bf4d552 Be more cautious about emitting COPY and RUN phrases while also supporting multiple COPY or RUN phrases if necessary. Output is the same. 2025-03-25 07:23:14 -06:00
Stephen Dennis
ab3796f3bf Initialize OutputStatus. 2025-03-25 07:23:14 -06:00
Stephen Dennis
b26fd96768 Update EastAsianWidth to Unicode 7.0.0 (and fix a trailing space bug). 2022-03-10 12:38:07 -07:00
brazilofmux
6aa4d2e0fd Fix Ubuntu compiler warnings. 2010-11-28 18:19:54 -08:00
brazilofmux
79c2ede4ec Avoid using negative initializers in UTF-8 run-copy table to fix gcc 3.4.2 and VC -W4 warnings (Issue 573). 2010-05-10 15:44:20 -07:00
Stephen Dennis
af46a2c701 Fix build break in utf on Debian GCC 4.4. 2010-02-03 23:11:56 -08:00
brazilofmux
15f9cddd0b State machine size was over-calculating by 256 bytes. 2009-02-01 15:16:36 -08:00
brazilofmux
25ea4afafc Use run-length encoded Unicode tables. 2009-02-01 13:40:38 -08:00
brazilofmux
645e08fc03 Run-length encode the state transition table. 2009-02-01 13:40:16 -08:00
brazilofmux
11f4912245 Remove ReportStatus(). 2009-01-31 16:41:12 -08:00
Brazil
1165f7d096 Fix build of ./utf directory on newer compilers. 2008-04-15 07:42:01 -07:00
brazilofmux
f52657b803 Generate separate copy/paste for stringutil.cpp and stringutil.h to make it less manual. 2008-04-12 22:20:42 -07:00
brazilofmux
34932af89f -- Build tr_tolower.txt and tr_toupper.txt from UnicodeMaster.
-- Support wider range of upper and lower case code points.

git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1652 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:17:54 -07:00
brazilofmux
c51ff824bb -- Change ReadCodePoint for classify.cpp to look only for first field.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1621 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:07:56 -07:00
brazilofmux
3e58cd243c -- Change input format of integers.cpp so that value is second column.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1619 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:07:20 -07:00
brazilofmux
e577fdd87c -- Define mux_iscolor() for determining whether a code point represents a color directive.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1362 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-08 07:21:53 -07:00
brazilofmux
689cb16f40 -- Fix transform and tr_utf8_latin1/ascii tables.
-- Add more error checking to smutil.cpp

git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_firan@1328 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 22:17:02 -07:00
brazilofmux
e28d98d370 -- Remove hardtabs and trailing spaces.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_firan@1285 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:56:40 -07:00
brazilofmux
e40b2eb8cf Add newline.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1182 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:15:58 -07:00
brazilofmux
1c958ba342 -- Format itt more nicely.
-- Add const to tables.
 -- Update VerifyTables().
 -- Add Printable.txt which containts the previous set of byte-sized printable characters.

git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1179 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:12:24 -07:00
brazilofmux
b7d4eef150 Finish feature-complete version.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1171 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 18:01:59 -07:00
brazilofmux
c17d08dddf Move ReadCodePoint() and DecodeCodePoint() to smutil.cpp.
Allow for more than two accepting states.

git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1166 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 18:00:22 -07:00
brazilofmux
0f661e0baa -- Make StateMachine class and pull most of the SM logic into it for use by classify and transform.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1163 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 17:59:19 -07:00