Update help text, build configs, and docs for unified date parser

- help.txt: Rewrite convtime() help to document all supported formats
  (legacy, ISO 8601, European order) with clear grammar descriptions.
  Update examples with verified epoch values.
- libmux.vcxproj, .filters: Replace timeparser.cpp with date_scan.cpp
- TOC.patchable (unix, win32): Replace timeparser.cpp with date_scan
- design-date-parser.md: Mark as Implemented, update file layout
- design-libmux-modules.md: timeparser → date_scan in module tables
- functions.cpp: Update fun_convtime comment to describe unified parser

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Stephen Dennis 2026-04-07 00:20:18 -06:00
parent 4884ba3dba
commit 17c545d883
8 changed files with 72 additions and 63 deletions

View file

@ -2,12 +2,13 @@
## Status
Proposed — replaces both `do_convtime()` and `ParseDate()` with a
single Ragel -G2 scanner + recursive descent parser.
Implemented (2026-04-06). Replaced both `do_convtime()` and
`ParseDate()` with a unified Ragel -G2 scanner + recursive descent
parser in `mux/lib/date_scan.rl`.
## Background
`convtime()` currently has two independent code paths:
`convtime()` previously had two independent code paths:
1. **`do_convtime()`** — Hand-written parser for the legacy format
`[Ddd] Mmm DD HH:MM:SS[.frac] YYYY`. Works correctly.
@ -246,37 +247,24 @@ Apr 6 26 ambiguous (year 26 vs day 26?)
## File Layout
```
mux/lib/date_scan.rl Ragel source (scanner)
mux/lib/date_scan.rl Ragel source (scanner + parser in one file)
mux/lib/date_scan.cpp Generated (read-only, chmod a-w)
mux/lib/date_parse.cpp Recursive descent parser
mux/lib/timeparser.cpp Deleted
mux/lib/timeutil.cpp do_convtime() removed
mux/include/timeutil.h ParseDate() signature unchanged or
replaced by date_parse()
mux/include/timeutil.h ParseDate() signature unchanged
```
Alternatively, the parser can live in the same `.rl` file after
the `%%write` block, keeping scanner and parser together.
The old `mux/lib/timeparser.cpp` (1650-line deduction engine) has
been deleted. `do_convtime()` remains in `timeutil.cpp` and is
still called by `CLinearTimeAbsolute::SetString()` as a first-try
fast path in `fun_convtime()`.
## Testing
- `testcases/convtime_fn.mux` — legacy format tests (must still pass)
- `testcases/parsedate_fn.mux` — update with ISO + name-based tests
- All 998+ smoke tests must continue to pass
- The round-trip `convtime(convsecs(N, utc), utc) == N` must hold
for all valid N
- `testcases/convtime_fn.mux` — legacy format (7 SHA1-based tests)
- `testcases/parsedate_fn.mux` — 30 tests: ISO 8601, free-form,
timezones, sub-seconds, boundaries, rejections
- All 998 smoke tests pass
## Migration
## Size
1. Write `date_scan.rl` (scanner) and parser.
2. Wire as the sole implementation of date parsing.
3. Remove `do_convtime()` and old `ParseDate()`.
4. Verify all smoke tests pass.
5. Update `help convtime` examples.
## Size Estimate
- Ragel scanner: ~150 lines of `.rl` (names are just alternations)
- Recursive descent parser: ~200 lines of C++
- Total: ~350 lines replacing ~1850 lines
(`do_convtime` ~130 + `ParseDate` ~1650 + helpers)
- `date_scan.rl`: 1277 lines (scanner + recursive descent parser)
- Replaces `timeparser.cpp`: 1653 lines (multi-pass deduction engine)

View file

@ -75,7 +75,7 @@ bootstraps new connections. Custom marshaling handles CStubSlaveProxy
| Mail & comsys | mail, comsys | ~10,500 |
| Objects & predicates | object, player, flags, predicates, player_c | ~9,000 |
| Configuration & logging | conf, log | ~4,200 |
| Queue & timing | cque, game, timer, timeutil, timeparser | ~8,000 |
| Queue & timing | cque, game, timer, timeutil, date_scan | ~8,000 |
| String & utility | stringutil, mathutil, wild, match, help | ~14,000 |
| Module system | modules, libmux, stubslave | ~4,200 |
| Unicode tables | utf8tables (generated) | ~11,000 |
@ -176,7 +176,7 @@ its first action, so existing code is unaffected.
| Category | Files |
|----------|-------|
| Crypto/hash | sha1, svdrand, svdhash |
| Time | timeutil, timeabsolute, timedelta, timeparser, timezone |
| Time | timeutil, timeabsolute, timedelta, date_scan, timezone |
| UTF-8 | utf8_collate, utf8_grapheme, utf8_normalize, utf8tables |
| String | stringutil |
| Math | mathutil, strtod |
@ -1265,7 +1265,7 @@ log (17), help (15), wild (8), modules (8), and others.
| Category | Files |
|---|---|
| UTF-8 / Unicode | utf8_collate, utf8_grapheme, utf8_normalize |
| Time utilities | timeabsolute, timedelta, timeparser, timeutil, timezone |
| Time utilities | timeabsolute, timedelta, date_scan, timeutil, timezone |
| Crypto / hash | sha1, svdhash, svdrand |
| Math | funmath, strtod |
| Network helpers | netaddr, telnet, slave |
@ -1304,7 +1304,7 @@ libmux.so requires either (a) creating a lighter header or (b) refactoring
| utf8_normalize | above + string_desc | config.h, stringutil.h, externs.h (for string_desc) |
| timeutil | UTF8, FIELDEDTIME, mux_isdigit, mux_atol | config.h, timeutil.h, stringutil.h |
| timeabsolute, timedelta | above + mux_sprintf, mux_assert | config.h, timeutil.h, stringutil.h |
| timeparser, timezone | FIELDEDTIME, UTF8 | config.h, timeutil.h |
| date_scan, timezone | FIELDEDTIME, UTF8 | config.h, timeutil.h |
| strtod | basic types only | config.h |
| netaddr, telnet | UTF8, socket types | config.h, system headers |
| alarm | std threading | config.h, system headers |

View file

@ -6854,21 +6854,35 @@ CONVTIME()
FUNCTION: convtime(<time string> [, <zonename>][, <precision>])
By default, this function converts a time string (of format
[Ddd] Mmm DD HH:MM:SS YYYY) into an integer which represents a
count of seconds since Jan 01 00:00:00 1970 UTC not including leap
secs where Ddd, if given, is the day of the week, Mmm is the month
name, DD is the day of the month, HH is the hour in 24-hour time,
MM is the minutes, SS is the seconds, and YYYY is the year.
Converts a time string into an integer representing seconds since
Jan 01 00:00:00 1970 UTC (not including leap seconds).
Keep in mind that the year may not be exactly four characters, and
it can also be negative. This server provides a proleptic Gregorian
calendar with a range of supported years between 27257 BC and
30826 AD. Year 0 is 1 BC, -1 is 2 BC, ..., so 27257 BC should be
given as -27256. This follows the astronomical conventions.
The following date formats are supported:
Note: The time string can also be free form and this function will
attempt to parse.
Legacy (convsecs output):
[Ddd] Mmm DD HH:MM:SS[.frac] YYYY
Mmm DD[,] YYYY HH:MM:SS[.frac] [TZ]
ISO 8601:
YYYY-MM-DDTHH:MM:SS[.frac][TZ] extended
YYYYMMDDTHHMMSS[.frac][TZ] basic (compact)
YYYY-DDDTHH:MM:SS[.frac][TZ] ordinal date
YYYYWww-DTHH:MM:SS[.frac][TZ] week date
European order:
DD Mmm YYYY HH:MM:SS[.frac] [TZ]
Where Ddd is an optional day-of-week, Mmm is a month name (full or
abbreviated), T can be the letter T or a space, and [.frac] is
optional fractional seconds with up to 7 digits (100ns resolution).
[TZ] can be: Z (UTC), +HHMM or -HH:MM (numeric offset), a named
timezone (UTC, GMT, EST, EDT, CST, CDT, MST, MDT, PST, PDT, etc.),
or a military single-letter timezone (A-Z except J).
The year can be negative. This server provides a proleptic Gregorian
calendar with years between 27257 BC and 30826 AD. Year 0 is 1 BC,
-1 is 2 BC, etc. (astronomical convention).
{ 'help convtime2' for more }
@ -6882,7 +6896,7 @@ CONVTIME() (continued)
utc - indicates that no timezone/DST conversions should be applied.
When used this way, there is a unique one-to-one relationship
between an integer and it's corresponding text-string which
between an integer and its corresponding text-string which
may be useful for handling other time zones and fictional
settings.
@ -6901,6 +6915,14 @@ CONVTIME() (continued)
You say, "709406574"
> say convtime(Wed Jun 24 10:22:54 1992, utc)
You say, "709381374"
> say convtime(2026-04-06T10:43:00Z)
You say, "1775472180"
> say convtime(2026-04-06T10:43:00-0700)
You say, "1775497380"
> say convtime(Apr 6 2026 10:43:00 UTC)
You say, "1775472180"
> say convtime(6 Apr 2026 10:43:00 +0000)
You say, "1775472180"
> say convtime(June 24th 1992 10:22:54.0001am Z)
You say, "709381374"
> say convtime(Wed 24 Jun 1992 10:22:54 -0700)
@ -6908,7 +6930,7 @@ CONVTIME() (continued)
> say convtime(2001-01-01T100000Q)
You say, "978357600"
> say convtime(-1605-120T123456M)
You say, "-112805738704"
You say, "-112805825104"
> say convtime(23456W12-1T123456T)
You say, "678039276896"

View file

@ -201,7 +201,7 @@
<Optimization Condition="'$(Configuration)|$(Platform)'=='Release|x64'">Full</Optimization>
<PreprocessorDefinitions Condition="'$(Configuration)|$(Platform)'=='Release|x64'">%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<ClCompile Include="lib\timeparser.cpp">
<ClCompile Include="lib\date_scan.cpp">
<Optimization Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">Disabled</Optimization>
<PreprocessorDefinitions Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">%(PreprocessorDefinitions)</PreprocessorDefinitions>
<BasicRuntimeChecks Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">EnableFastChecks</BasicRuntimeChecks>

View file

@ -50,7 +50,7 @@
<ClCompile Include="lib\timedelta.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="lib\timeparser.cpp">
<ClCompile Include="lib\date_scan.cpp">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="lib\timeutil.cpp">

View file

@ -720,20 +720,17 @@ static FUNCTION(fun_convsecs)
// ---------------------------------------------------------------------------
// fun_convtime.
//
// With one argument, it converts a local time string in the format
//'[Ddd] Mmm DD HH:MM:SS YYYY' to a count of seconds from Jan 01 00:00:00 1970
// UTC.
// Converts a date/time string to epoch seconds. Two parsing paths are
// tried in order:
//
// If a second argument is given, it is the <zonename>:
// 1. SetString() — legacy format: [Ddd] Mmm DD HH:MM:SS[.frac] YYYY
// 2. ParseDate() — unified Ragel scanner + recursive descent parser
// handling ISO 8601, month-name, European order, timezones, etc.
//
// local - indicates that the given time string is for the local timezone
// local DST adjustments (default if no second argument is given).
// If <zonename> is "utc", no local timezone conversion is applied.
// Otherwise, times without an explicit timezone are treated as local.
//
// utc - indicates that no timezone/DST conversions should be applied.
// This is useful to give a unique one-to-one mapping between an
// integer and it's corresponding text-string.
//
// This function returns -1 if there was a problem parsing the time string.
// <precision> controls fractional seconds in the output.
//
static FUNCTION(fun_convtime)
{

View file

@ -183,7 +183,8 @@ lib/svdhash.cpp
lib/svdrand.cpp
lib/timeabsolute.cpp
lib/timedelta.cpp
lib/timeparser.cpp
lib/date_scan.cpp
lib/date_scan.rl
lib/timeutil.cpp
lib/timezone.cpp
lib/utf8_collate.cpp

View file

@ -165,7 +165,8 @@ lib/svdhash.cpp
lib/svdrand.cpp
lib/timeabsolute.cpp
lib/timedelta.cpp
lib/timeparser.cpp
lib/date_scan.cpp
lib/date_scan.rl
lib/timeutil.cpp
lib/timezone.cpp
lib/utf8_collate.cpp