Stephen Dennis
14541d5335
Backport Unicode fixes: pairs DFA, collation NFC tiebreaker, tool updates
...
Port three fixes from libutf/master:
1. Fix pairs DFA generator: always resolve undefined transitions
before optimization. The old code only called SetUndefinedStates
when -d was passed, causing NFC over-composition and DUCET
contraction false-matches.
2. Regenerate NFC compose tables (129→1010 states) and DUCET
contraction tables (27→294 states) with the corrected generator.
3. NFC tiebreaker in collation: normalize to NFC before binary
comparison so canonically equivalent strings compare equal per UCA.
Also updates all four table generator tools (classify, integers,
strings, pairs) to support -o/-i flags for output file control,
and adds LIBMUX_API compat define to Makefile.in preamble.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:44:40 -06:00
Stephen Dennis
dbd7cb4267
Use nullptr instead of NULL.
2025-03-30 09:10:10 -06:00
Stephen Dennis
b92a76c0e9
Small clean up of smutil.cpp.
2025-03-30 07:27:12 -06:00
Stephen Dennis
2a9061df5b
COPY phase can be -128
2025-03-25 07:23:14 -06:00
Stephen Dennis
429bf4d552
Be more cautious about emitting COPY and RUN phrases while also supporting multiple COPY or RUN phrases if necessary. Output is the same.
2025-03-25 07:23:14 -06:00
Stephen Dennis
ab3796f3bf
Initialize OutputStatus.
2025-03-25 07:23:14 -06:00
Stephen Dennis
b26fd96768
Update EastAsianWidth to Unicode 7.0.0 (and fix a trailing space bug).
2022-03-10 12:38:07 -07:00
brazilofmux
6aa4d2e0fd
Fix Ubuntu compiler warnings.
2010-11-28 18:19:54 -08:00
brazilofmux
79c2ede4ec
Avoid using negative initializers in UTF-8 run-copy table to fix gcc 3.4.2 and VC -W4 warnings (Issue 573).
2010-05-10 15:44:20 -07:00
Stephen Dennis
af46a2c701
Fix build break in utf on Debian GCC 4.4.
2010-02-03 23:11:56 -08:00
brazilofmux
15f9cddd0b
State machine size was over-calculating by 256 bytes.
2009-02-01 15:16:36 -08:00
brazilofmux
25ea4afafc
Use run-length encoded Unicode tables.
2009-02-01 13:40:38 -08:00
brazilofmux
645e08fc03
Run-length encode the state transition table.
2009-02-01 13:40:16 -08:00
brazilofmux
11f4912245
Remove ReportStatus().
2009-01-31 16:41:12 -08:00
Brazil
1165f7d096
Fix build of ./utf directory on newer compilers.
2008-04-15 07:42:01 -07:00
brazilofmux
f52657b803
Generate separate copy/paste for stringutil.cpp and stringutil.h to make it less manual.
2008-04-12 22:20:42 -07:00
brazilofmux
34932af89f
-- Build tr_tolower.txt and tr_toupper.txt from UnicodeMaster.
...
-- Support wider range of upper and lower case code points.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1652 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:17:54 -07:00
brazilofmux
c51ff824bb
-- Change ReadCodePoint for classify.cpp to look only for first field.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1621 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:07:56 -07:00
brazilofmux
3e58cd243c
-- Change input format of integers.cpp so that value is second column.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1619 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-09 15:07:20 -07:00
brazilofmux
e577fdd87c
-- Define mux_iscolor() for determining whether a code point represents a color directive.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1362 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-08 07:21:53 -07:00
brazilofmux
689cb16f40
-- Fix transform and tr_utf8_latin1/ascii tables.
...
-- Add more error checking to smutil.cpp
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_firan@1328 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 22:17:02 -07:00
brazilofmux
e28d98d370
-- Remove hardtabs and trailing spaces.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_firan@1285 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:56:40 -07:00
brazilofmux
e40b2eb8cf
Add newline.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1182 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:15:58 -07:00
brazilofmux
1c958ba342
-- Format itt more nicely.
...
-- Add const to tables.
-- Update VerifyTables().
-- Add Printable.txt which containts the previous set of byte-sized printable characters.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1179 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 21:12:24 -07:00
brazilofmux
b7d4eef150
Finish feature-complete version.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1171 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 18:01:59 -07:00
brazilofmux
c17d08dddf
Move ReadCodePoint() and DecodeCodePoint() to smutil.cpp.
...
Allow for more than two accepting states.
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1166 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 18:00:22 -07:00
brazilofmux
0f661e0baa
-- Make StateMachine class and pull most of the SM logic into it for use by classify and transform.
...
git-svn-id: https://tinymux.googlecode.com/svn/branches/dev_brazil@1163 d1b986fa-651c-0410-a323-35a8662cf44d
2007-05-07 17:59:19 -07:00