move struct Waif and WAIF_MAPSZ to waif.h (C allows forward references)
fix include orderings in waif.{h,c}
register_waif() needs to be at the bottom of the file
This commit is the substantive portion of the Waif implementation.
The combination of this commit with its parent
(add back waif-related cvs log comments)
is intended to match either of the differences:
codepoint/unicode (51761cdc98)
..codepoint/unicode-waif (8f892adbf7b6a5973c3268942087584eb01242f1)
codepoint/unicode-xml (67aa153f76bcdbd9917bdf57fd380d9e7ffb419e)
..codepoint/unicode-waif-xml( c3d6fa5e58e358a0a75662a6ddaaecab4d39f306)
(these are both essentially the same) except for those files for which
the only differences are in the CVS log entries.
This version is purposefully broken (in order that the differences
line up as well as possible), notably in execute.c
(OP_INDEXSET,OP_REF,OP_CALL_VERB).
Add in bitwise AND ".&.", OR ".|.", and XOR ".^."
Add arithmetic left shift "<<", arithmetic right shift ">>",
and logical right shift operators ">>>"
This retains the original parsing of something like "9.^.5" to be
interpreted as "square root of 9" However, "9.^. 5" is interpreted as
"9 XOR 5".
( The number parsing code moving to parse_number() is the only
significant change in adapting this extension to the new server;
the original behavior of the extension is retained in this commit
--wrog )
Change-Id: I99291bed3d07d68ced204ea27bf684dc09a861fc
Phase one = use the various API calls everywhere, but define them all
as inline routines that perform the straight-ASCII functions we were
doing before, so as to make this is a grand no-op to re create the
non-Unicode server we already had previously.
Important differences from the original implementation:
(1)
In this timeline, HPA's libucd (Unicode Character Database) and the
inclusion of a subdirectory to build it in (ucd/) NEVER HAPPENED;
In phase two, we'll instead go straight to an implementation using
GNU libunistring as a shared lib, only (re-)introducing libucd and
libicuuc as alternatives later.
(...in other news we have *always* been at war with Eastasia.
Also, Bobby Ewing is dead; as you may recall, Dallas received a
long-overdue cancellation at the end of its 8th season.
Alsoalso, Skywalker Ranch was struck by a meteor in 1981,
so the Star Wars series ended with The Empire Strikes Back;
Ron Moore is currently busy on a reboot/reimagining....)
(2)
There is no skip_utf(). Instead, we have
(*) utf_byte_index(),
a re-arranged version that is more explicit about
the mission of converting character indices to byte indices
and can be used prior to any range checking,
(*) utf_byte_range()
converts *pairs* of char indices to byte indices
since this seems to come up a lot.
(*) utf_char_index(),
goes the other way, but with fewer traversals
(3)
my-ctype.h retains its role of working around mistakes in ctype.h on
broken systems. All changes you might have thought you made there are
actually in utf-ctype.h, which, oddly enough serves as the header file
for the new functions being introduced in utf-ctype.c
Original commits cannibalized from the codepoint branch:
. Initial work on Unicode compatibility
-- hpa
. fix ok_identifier (was: Correct which ctype to use ...)
revised to leave my-ctype.h reference in db_io.c alone
since it is solely there to provide (ascii) isspace().
-- hpa
. Kenny's fixes to get it to load Minimal.db and JHCore. (2)
-- james <james@nanaka.home.place.org>
. Slew a few bugs, made string handling unicode-y.
--james <james@nanaka.home.place.org>
. make NET_SINGLE do utf8
--wrog <crew@cs.stanford.edu>
(since, among other things, fixing net_single.c
should have been done at the same time.)
. Don't consider U+007F or surrogates to be printable
--Rob Leslie <rob@mars.org>
(changes my_is_printable())
This commit alters my_is_printable() to reject surrogate code
points, in part so that bf_tochar() doesn't end up returning an
empty string instead of generating an error. (Surrogates are
already rejected by put_utf().)
. Caused online programming of verbs containing unicode to stop almost
working for the wrong reasons and, along with reading of verbs
containing unicode from saved databases, start working for the right
reasons.
--james <james@nanaka.home.place.org>
(adds a number of stream_add_utf() uses to parser.y --wrog)
. Let stream_add_utf() return the result of put_utf()
--Rob Leslie <rob@mars.org>
There's no point duplicating all the validity checks in put_utf()
elsewhere.
scanf has too many annoying behaviors so we are replacing it.
E.g., Pavel was unhappy about %* specs not being counted in the
return value and was having to add 'dummy' variables all over.
I myself am unhappy about how you can put a \n at the end of a pattern
giving the false sense that you want it to match a newline when in
fact there's no guarantee of that at all, or that, in ANY pattern with
characters after the last % spec that you'll have any idea whether
those characters were reached or not (hence the need for all of these
%c-at-the-end bullshit patterns)
But worst of all, you cannot count on E_RANGE errors being raised when
they're needed because there's disagreement about when that should
happen, putting this in the realm of undefined behavior (hence certain
vendors putting deprecation warnings on %d and all of the other integer
conversions)
So.., fuck it. We write our own function from the ground up that does
what we need (and very little more) and will thence be completely
portable (knock wood).
Well okay, it still relies on strtoimax() continuing to behave sensibly.
We'll see how that goes.
--wrog
Every dbio_read_*() returns status and uses a return argument.
...makes it much harder to *not* check success/failure.
We need to be clear about what we expect to see in the db file.
Accepting out-of-range integers leads to silent corruption of data.
Some new rules:
Do not try to make use of [INTMAX_MAX+1..UINTMAX_MAX]; all integers we
care about are in the intmax_t range. If something needs to be
unsigned, fine; it's in [0..INTMAX_MAX]
Do all integer range checks in the intmax_t world to avoid surprises.
We need to consolidate line reading in one place,
remove pointless optimizations (Yes, the " recycled" line is less
than 20 characters, but is using the stream *that* expensive?
Also I'm done with having static streams all over the place)
and make it easier to introduce EOF-checking later.
dbio_read_line()
= internal/static routine that does the actual work
dbio_read_string_temp <- dbio_read_string
= public routine for non-persistent strings
dbio_read_string_intern
= public routine for persistent strings
this subphase: Do all of the int(32_t)->Num retypings.
!(- use C99-like macros for printf)
!(- use [u]int*_t types (autoconf can verify they exist))
- abstract Num and Objid out
- default to 64 bit integers
!(- un-pointerify floating-point numbers (makes no sense on modern hardware))
this subphase: Fix all of the printf/scanf format strings.
!(- use C99-like macros for printf)
!(- use [u]int*_t types (autoconf can verify they exist))
- abstract Num and Objid out
- default to 64 bit integers
!(- un-pointerify floating-point numbers (makes no sense on modern hardware))
- - from james
Kenny's fixes to get it to load Minimal.db and JHCore. (1)
(evidently Kenny found two PRI*N goofs in db_file.c --wrog)
(yes, we chased down all of the places where task ids can live).
For now, this is a no-op, but having identified all of the places
where they occur, we can range-check them later.
(phase 1 = changes having nothing to do with introducing [u]int*_t,
missing headers and adding/fixing function prototypes,
Num-introduction, float-unboxing, autoconf, or the new math builtins
here: everything having to do with db file reading/writing
--wrog)
was:
- use C99-like macros for printf
- use [u]int*_t types (autoconf can verify they exist)
- abstract Num and Objid out
- default to 64 bit integers
- un-pointerify floating-point numbers (makes no sense on modern hardware)
For Unicode, the TYPE_LIST and TYPE_STR cases for these operators
will need to be handled separately, so I'm separating them *now*.
Also took the opportunity to restructure their error checking
(to remove redundant tests and unnecessarily repeated blocks
of free_var() calls).
Also correspondingly changed the API on sublist(), listrangeset(),
substr(), strrangeset() -- going forward, it will turn out far easier
if these functions continue to take byte indicies rather than
character indices and instead do the translation *before* calling
them. That way they won't need to be modified at all for Unicode.
It also turns out that using 'from' and 'after' parameters rather than
'from' and 'to' will be *way* more convenient in Unicode Land (see,
a byte index is most conveniently identified with the *start* of
the respective character, so to get a proper byte range, what you
need is the start of the first character *following* the range;
subtract 1 and you're done; being instead given the start of the
last character of the range is a bit more complicated and a PITA.
Also took the opportunity to replace character twiddling loops in the
list.c routines with memcpy() + reordering them so that (maybe) we get
a bit less cache-miss buggery.
Time to impose some discipline on #include header ordering:
(1) We need to be absolutely certain everybody is seeing
the SAME #define settings from config.h and options.h
(so many of the my_foo.h depend on config.h settings
and yet config.h was being listed later. how did this happen?)
(2) It REALLY helps for documentation and code comprehension
to have .c files clearly point to the .h files that
export their interfaces (usually it's the like-named .h
but sometimes it is not and this can be confusing to
the newbie maintainer)
So... new rules:
For .h files:
config.h if referenced, goes FIRST.
options.h if referenced, goes SECOND.
For .c files the ordering of #includes shall be:
(1) The .h file(s) defining the exported interfaces
that this .c file is implementing
(typically this will be just one file, foo.h
if this is foo.c, but exceptions exist)
(2) config.h, then options.h, in a stanza by themselves
if referenced and not already included by (1)
For both .c and .h files we then have two more stanzas:
(3) system headers,
including the "my-foo.h" corrective headers.
(4) own headers, other than the ones that go in (1)
Except for special cases like net_proto.c and net_mplex.c
where the whole purpose of the file is to BE an #include festival,
there should be no further #includes after the initial block.
All includes should appear at the top of the file where we can see them.
... ok, I'm done.
Introduces the options.h #defines, the server.h SVO_ constants,
and the canonicalization/caching code; nothing actually pays
attention to their values yet.
Granted, the programmer's manual only says it has to prevent reads,
but it's decidedly wacky to have, e.g., $server_options.protect_name
set, and a nonwizard viewing .name or doing .name[1]="a" or
.name[1..1]="a" getting errors, while .name="fred" sails through
just fine. You can tell this code got used a lot.
