mmo-ref/tinymux
1
0
Fork 0
mirror of https://github.com/brazilofmux/tinymux synced 2026-08-13 00:23:11 -04:00
Code Issues Projects Releases Packages Wiki Activity

dbload: mistake-proof the SQLite import path (#766)

Browse source 
Migrating an existing game to the SQLite backend had two silent traps that
both surfaced as "old passwords/characters don't work":

  1. dbconvert resolves <basename>.sqlite relative to cwd, but the server
     reads data/<name>.sqlite. Running db_load from the wrong directory put
     the database where the server never looks, so old characters appeared
     to vanish while a fresh login still worked against the stock db.
  2. A populated netmux.sqlite silently shadows the netmux.db flatfile at
     boot, so dropping in an old flatfile did nothing.

Changes:
  - db_load/db_unload are now cwd-independent: the .sqlite always lands in
    the game's data/ dir (next to the script), file args are resolved
    against the caller's dir, and the scripts echo the absolute path. Arg
    handling is space-safe (set --) and POSIX sh.
  - dbconvert prints the exact database file it opened (CSQLiteDB::GetPath).
  - The "would overwrite" guard now names the file and gives two ways
    forward; a new -f/force option lets a load replace an existing db (the
    load already clears attributes/objects/attr-names cleanly).
  - The server logs a line when it warm-starts from SQLite and the flatfile
    was not consulted, making the precedence visible.
  - New docs/importing-a-game.md documents the two-file model and import
    steps.

Verified: db_load from an unrelated dir lands the .sqlite in data/; the
guard refuses without -f and replaces with -f; db_unload round-trip is
byte-identical (passwords preserved). Smoke: 1078 ok / 0 new failures
(TC001/TC009 pre-existing on master).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Stephen Dennis 2026-06-07 13:23:12 -05:00
parent 982b4713e6
commit ecae4df9de
8 changed files with 266 additions and 37 deletions
Download patch file Download diff file Expand all files Collapse all files

85
docs/importing-a-game.md Normal file
View file

@ -0,0 +1,85 @@
# Importing an existing game / migrating to the SQLite backend
TinyMUX now stores the live game in a **SQLite database** rather than reading
the flatfile directly on every boot. This changes how you import an existing
game, and it is the source of a common "my old passwords don't work" symptom
when migrating from an older build. This page explains the model and the steps.
## The two files in `game/data/`
| File | What it is | Role |
|------|------------|------|
| `netmux.db` | The **flatfile** (human-readable text, starts with `+X...`) | Seed / export format only |
| `netmux.sqlite` | The **live database** the server actually runs from | Authoritative |
The SQLite filename is derived from your configured `input_database`
(in `netmux.conf`) by replacing the `.db` suffix with `.sqlite`. So
`input_database data/netmux.db` means the server reads and writes
`data/netmux.sqlite`.
## Boot precedence (read this twice)
> **If `data/netmux.sqlite` exists and contains a game, the server warm-starts
> from it and the `netmux.db` flatfile is never read.**
On a brand-new install, the first boot builds `netmux.sqlite` from the seed
flatfile `netmux.db`. After that, the flatfile is ignored. This is why simply
dropping your old game in as `netmux.db` does nothing once a `.sqlite` exists —
you keep logging into whatever is already in the SQLite database. The startup
log notes this with a "Warm-started from SQLite database; the flatfile ... was
not consulted" line.
## Importing your old flatfile
You have two equivalent options. Both assume the server is **stopped**.
### Option A — let the server rebuild from a flatfile
```sh
cd game
rm -f data/netmux.sqlite # remove the stale/seed database
cp /path/to/your_old_game.flat data/netmux.db
./bin/netmux # first boot rebuilds netmux.sqlite from it
```
### Option B — load the flatfile explicitly with db_load
```sh
cd game/data
./db_load netmux /path/to/your_old_game.flat
cd ..
./bin/netmux
```
`db_load` always writes the `.sqlite` into the game's `data/` directory (next to
the script), regardless of which directory you run it from, so it can no longer
land somewhere the server doesn't read. It prints the exact file it loaded into.
If `data/netmux.sqlite` already exists, `db_load` refuses to overwrite it and
tells you the file path. Either remove that file first, or force the replacement:
```sh
./db_load -f netmux /path/to/your_old_game.flat
```
To bring across comsys and mail as well:
```sh
./db_load netmux your_old_game.flat -C comsys.db -m mail.db
```
## Exporting back to a flatfile
```sh
cd game/data
./db_unload netmux backup.flat # writes backup.flat in this directory
./db_unload netmux backup.flat -C comsys.db -m mail.db
```
## Passwords
Password hashes (`$SHA1$...`, `$1$...`, etc.) are carried through import
unchanged, so old passwords continue to work once you are logging into the
correct database. If old passwords appear to fail, the cause is almost always
that the server is reading a different `netmux.sqlite` than the one your game
was imported into — see the boot precedence above.

77
mux/game/data/db_load
View file

@ -2,25 +2,74 @@
#
# Load a MUX flatfile into the SQLite database.
#
# Usage: db_load <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]
# Usage: db_load [-f] <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]
#
# Example: db_load netmux netmux.flat
# db_load netmux netmux.flat -C comsys.db -m mail.db
#
# The SQLite database (<basename>.sqlite) is always written next to this
# script, i.e. into the game's data/ directory, so it matches the server's
# configured input_database no matter which directory you run this from.
# Flatfile arguments are resolved relative to your current directory.
#
# Use -f to replace an existing SQLite database (the load otherwise refuses
# to overwrite a database that already contains a game).
#
BIN=../bin
# Directory this script lives in (the game's data/ directory).
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
LD_LIBRARY_PATH=$BIN; export LD_LIBRARY_PATH
DYLD_LIBRARY_PATH=$BIN; export DYLD_LIBRARY_PATH
# Directory the caller invoked us from, for resolving relative file arguments.
INVOKE_DIR=$(pwd)
case $# in
2) $BIN/dbconvert -d "$1" -l -i "$2" ;;
4) $BIN/dbconvert -d "$1" -l -i "$2" $3 "$4" ;;
6) $BIN/dbconvert -d "$1" -l -i "$2" $3 "$4" $5 "$6" ;;
*) echo "Usage: $0 <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]"
echo " e.g. $0 netmux netmux.flat"
echo " $0 netmux netmux.flat -C comsys.db -m mail.db"
exit 1 ;;
esac
abspath() {
case "$1" in
/*) printf '%s\n' "$1" ;;
*) printf '%s\n' "$INVOKE_DIR/$1" ;;
esac
}
exit 0
usage() {
echo "Usage: $0 [-f] <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]"
echo " e.g. $0 netmux netmux.flat"
echo " $0 netmux netmux.flat -C comsys.db -m mail.db"
echo " -f Replace an existing SQLite database."
exit 1
}
FORCE=
if [ "$1" = "-f" ]; then
FORCE=-f
shift
fi
[ $# -ge 2 ] || usage
BASENAME="$1"
FLATFILE=$(abspath "$2")
shift 2
COMSYS=
MAIL=
while [ $# -ge 1 ]; do
case "$1" in
-C) [ $# -ge 2 ] || usage; COMSYS=$(abspath "$2"); shift 2 ;;
-m) [ $# -ge 2 ] || usage; MAIL=$(abspath "$2"); shift 2 ;;
*) usage ;;
esac
done
BIN="$SCRIPT_DIR/../bin"
LD_LIBRARY_PATH="$BIN"; export LD_LIBRARY_PATH
DYLD_LIBRARY_PATH="$BIN"; export DYLD_LIBRARY_PATH
cd "$SCRIPT_DIR" || exit 1
echo "Loading into: $SCRIPT_DIR/$BASENAME.sqlite"
set -- -d "$BASENAME" -l -i "$FLATFILE"
[ -n "$FORCE" ] && set -- "$@" -f
[ -n "$COMSYS" ] && set -- "$@" -C "$COMSYS"
[ -n "$MAIL" ] && set -- "$@" -m "$MAIL"
"$BIN/dbconvert" "$@"
exit $?

64
mux/game/data/db_unload
View file

@ -7,22 +7,60 @@
# Example: db_unload netmux netmux.flat
# db_unload netmux netmux.flat -C comsys.db -m mail.db
#
# The SQLite database (<basename>.sqlite) is always read from next to this
# script, i.e. from the game's data/ directory, so it matches the server's
# configured input_database no matter which directory you run this from.
# Output flatfile arguments are resolved relative to your current directory.
#
# Note: The server should not be running during export.
#
BIN=../bin
# Directory this script lives in (the game's data/ directory).
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
LD_LIBRARY_PATH=$BIN; export LD_LIBRARY_PATH
DYLD_LIBRARY_PATH=$BIN; export DYLD_LIBRARY_PATH
# Directory the caller invoked us from, for resolving relative file arguments.
INVOKE_DIR=$(pwd)
case $# in
2) $BIN/dbconvert -d "$1" -u -o "$2" ;;
4) $BIN/dbconvert -d "$1" -u -o "$2" $3 "$4" ;;
6) $BIN/dbconvert -d "$1" -u -o "$2" $3 "$4" $5 "$6" ;;
*) echo "Usage: $0 <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]"
echo " e.g. $0 netmux netmux.flat"
echo " $0 netmux netmux.flat -C comsys.db -m mail.db"
exit 1 ;;
esac
abspath() {
case "$1" in
/*) printf '%s\n' "$1" ;;
*) printf '%s\n' "$INVOKE_DIR/$1" ;;
esac
}
exit 0
usage() {
echo "Usage: $0 <basename> <flatfile> [-C <comsys.db>] [-m <mail.db>]"
echo " e.g. $0 netmux netmux.flat"
echo " $0 netmux netmux.flat -C comsys.db -m mail.db"
exit 1
}
[ $# -ge 2 ] || usage
BASENAME="$1"
FLATFILE=$(abspath "$2")
shift 2
COMSYS=
MAIL=
while [ $# -ge 1 ]; do
case "$1" in
-C) [ $# -ge 2 ] || usage; COMSYS=$(abspath "$2"); shift 2 ;;
-m) [ $# -ge 2 ] || usage; MAIL=$(abspath "$2"); shift 2 ;;
*) usage ;;
esac
done
BIN="$SCRIPT_DIR/../bin"
LD_LIBRARY_PATH="$BIN"; export LD_LIBRARY_PATH
DYLD_LIBRARY_PATH="$BIN"; export DYLD_LIBRARY_PATH
cd "$SCRIPT_DIR" || exit 1
echo "Exporting from: $SCRIPT_DIR/$BASENAME.sqlite"
set -- -d "$BASENAME" -u -o "$FLATFILE"
[ -n "$COMSYS" ] && set -- "$@" -C "$COMSYS"
[ -n "$MAIL" ] && set -- "$@" -m "$MAIL"
"$BIN/dbconvert" "$@"
exit $?

2
mux/include/modules.h
View file

@ -844,7 +844,7 @@ public:
//
virtual MUX_RESULT DbConvert(const UTF8 *infile, const UTF8 *outfile,
const UTF8 *basename, bool bCheck, bool bLoad, bool bUnload,
const UTF8 *comsys_file, const UTF8 *mail_file) = 0;
const UTF8 *comsys_file, const UTF8 *mail_file, bool bForce) = 0;
// Query the static configuration basket. The driver calls this once
// after LoadGame to get a snapshot of mudconf values it needs.

9
mux/include/sqlitedb.h
View file

@ -15,6 +15,7 @@
#include <cstddef>
#include <cstdint>
#include <functional>
#include <string>
// When building the standalone test harness, define TINYMUX_TYPES_DEFINED
// via compiler flags and provide these types. When building inside TinyMUX,
@ -40,6 +41,10 @@ public:
void Close();
bool IsOpen() const { return nullptr != m_db; }
// Path of the currently-open database file (empty when closed).
//
const char *GetPath() const { return m_path.c_str(); }
// Object metadata operations.
// These correspond to the db[] array and s_Location() etc.
//
@ -294,6 +299,10 @@ public:
private:
sqlite3 *m_db;
// Path of the open database file, for diagnostics.
//
std::string m_path;
// Object metadata statements.
//
sqlite3_stmt *m_stmtObjInsert;

39
mux/modules/engine/engine_com.cpp
View file

@ -2610,7 +2610,7 @@ public:
virtual MUX_RESULT Shutdown(void);
virtual MUX_RESULT DbConvert(const UTF8 *infile, const UTF8 *outfile,
const UTF8 *basename, bool bCheck, bool bLoad, bool bUnload,
const UTF8 *comsys_file, const UTF8 *mail_file);
const UTF8 *comsys_file, const UTF8 *mail_file, bool bForce);
virtual MUX_RESULT GetConfig(DRIVER_CONFIG *pConfig);
virtual MUX_RESULT MarkConnected(dbref player);
virtual MUX_RESULT DumpChildExited(int child_pid);
@ -2984,6 +2984,16 @@ MUX_RESULT CGameEngine::LoadGame(const UTF8 *configFile,
//
bDoFlatfileLoad = false;
bLoadedGameFromSQLite = true;
// Make the precedence visible: a populated SQLite database
// shadows the configured flatfile, so a freshly dropped-in
// flatfile would otherwise appear to be silently ignored.
//
STARTLOG(LOG_ALWAYS, "INI", "LOAD")
log_text(T("Warm-started from SQLite database; the flatfile "));
log_text(mudconf.indb);
log_text(T(" was not consulted. Remove the SQLite database to reload from a flatfile."));
ENDLOG
}
else if (sqlite_load_rc < 0)
{
@ -3660,7 +3670,7 @@ static void dbconvert_info(int fmt, int flags, int ver)
MUX_RESULT CGameEngine::DbConvert(const UTF8 *infile, const UTF8 *outfile,
const UTF8 *basename, bool bCheck, bool bLoad, bool bUnload,
const UTF8 *comsys_file, const UTF8 *mail_file)
const UTF8 *comsys_file, const UTF8 *mail_file, bool bForce)
{
int setflags, clrflags, ver;
int db_ver, db_format, db_flags;
@ -3709,11 +3719,30 @@ MUX_RESULT CGameEngine::DbConvert(const UTF8 *infile, const UTF8 *outfile,
mux_fprintf(stderr, T("Can\xE2\x80\x99t open SQLite database.\n"));
return MUX_E_FAIL;
}
else if (cc == HF_OPEN_STATUS_OLD)
// Report exactly which file we opened so it is never a mystery where the
// database lives relative to the server's configured input_database.
//
const char *pDbPath = g_pSQLiteBackend->GetDB().GetPath();
mux_fprintf(stderr, T("Database file: %s\n"),
(nullptr != pDbPath && '\0' != pDbPath[0]) ? pDbPath : "(unknown)");
if (cc == HF_OPEN_STATUS_OLD)
{
if (setflags == OUTPUT_FLAGS)
if (setflags == OUTPUT_FLAGS && !bForce)
{
mux_fprintf(stderr, T("Would overwrite existing SQLite database.\n"));
// The target SQLite database already holds a game. Loading would
// replace it, so refuse unless the caller explicitly forces it.
// Name the file and spell out both ways forward rather than
// leaving a dead-end that tempts running from the wrong directory.
//
mux_fprintf(stderr,
T("Refusing to overwrite the existing SQLite database:\n"
" %s\n"
"That file already contains a game. To replace it, either:\n"
" * remove the file shown above and re-run, or\n"
" * re-run this load with the -f (force) option.\n"),
(nullptr != pDbPath && '\0' != pDbPath[0]) ? pDbPath : "(unknown)");
CLOSE;
return MUX_E_FAIL;
}

9
mux/modules/engine/sqlitedb.cpp
View file

@ -132,6 +132,14 @@ bool CSQLiteDB::Open(const char *path)
return false;
}
// Remember the path we opened so callers can report exactly which file
// backs the database.
//
if (nullptr != path)
{
m_path.assign(path);
}
return true;
}
@ -143,6 +151,7 @@ void CSQLiteDB::Close()
sqlite3_close(m_db);
m_db = nullptr;
}
m_path.clear();
}
bool CSQLiteDB::ConfigurePragmas()

18
mux/src/driver.cpp
View file

@ -37,6 +37,7 @@ static bool standalone_load = false;
static bool standalone_unload = false;
static const UTF8 *standalone_comsys_file = nullptr;
static const UTF8 *standalone_mail_file = nullptr;
static bool standalone_force = false;
// dbconvert delegates to engine via mux_IGameEngine::DbConvert.
//
@ -61,7 +62,8 @@ static void dbconvert(void)
mr = pEngine->DbConvert(standalone_infile, standalone_outfile,
standalone_basename, standalone_check, standalone_load,
standalone_unload, standalone_comsys_file, standalone_mail_file);
standalone_unload, standalone_comsys_file, standalone_mail_file,
standalone_force);
pEngine->Release();
exit(MUX_SUCCEEDED(mr) ? 0 : 1);
}
@ -157,6 +159,7 @@ void init_sql(void)
#define CLI_DO_ERRORPATH CLI_USER+11
#define CLI_DO_COMSYS_FILE CLI_USER+12
#define CLI_DO_MAIL_FILE CLI_USER+13
#define CLI_DO_FORCE CLI_USER+14
static bool bMinDB = false;
static bool bSyntaxError = false;
@ -182,7 +185,8 @@ static CLI_OptionEntry OptionTable[] =
{ "C", CLI_REQUIRED, CLI_DO_COMSYS_FILE },
{ "m", CLI_REQUIRED, CLI_DO_MAIL_FILE },
{ "p", CLI_REQUIRED, CLI_DO_PID_FILE },
{ "e", CLI_REQUIRED, CLI_DO_ERRORPATH }
{ "e", CLI_REQUIRED, CLI_DO_ERRORPATH },
{ "f", CLI_NONE, CLI_DO_FORCE }
};
static void CLI_CallBack(CLI_OptionEntry *p, const char *pValue)
@ -256,6 +260,11 @@ static void CLI_CallBack(CLI_OptionEntry *p, const char *pValue)
standalone_mail_file = reinterpret_cast<const UTF8 *>(pValue);
break;
case CLI_DO_FORCE:
g_bStandAlone = true;
standalone_force = true;
break;
case CLI_DO_USAGE:
default:
bSyntaxError = true;
@ -346,13 +355,14 @@ int DCL_CDECL main(int argc, char *argv[])
mux_fprintf(stderr, T("Version: %s" ENDLINE), g_version);
if (g_bStandAlone)
{
mux_fprintf(stderr, T("Usage: %s -d <dbname> [-i <infile>] [-o <outfile>] [-l|-u|-k] [-C <comsys>] [-m <mail>]" ENDLINE), pProg);
mux_fprintf(stderr, T(" -d Basename." ENDLINE));
mux_fprintf(stderr, T("Usage: %s -d <dbname> [-i <infile>] [-o <outfile>] [-l|-u|-k] [-f] [-C <comsys>] [-m <mail>]" ENDLINE), pProg);
mux_fprintf(stderr, T(" -d Basename (the SQLite file is <basename>.sqlite, relative to the current directory)." ENDLINE));
mux_fprintf(stderr, T(" -i Input file." ENDLINE));
mux_fprintf(stderr, T(" -k Check." ENDLINE));
mux_fprintf(stderr, T(" -l Load (import flatfile into SQLite)." ENDLINE));
mux_fprintf(stderr, T(" -o Output file." ENDLINE));
mux_fprintf(stderr, T(" -u Unload (export SQLite to flatfile)." ENDLINE));
mux_fprintf(stderr, T(" -f Force load over an existing SQLite database (replaces it)." ENDLINE));
mux_fprintf(stderr, T(" -C Comsys flatfile (import/export)." ENDLINE));
mux_fprintf(stderr, T(" -m Mail flatfile (import/export)." ENDLINE));
}