Add undiscovered room hints with per-character known-room caching

docs/undiscovered-rooms.md

@@ -0,0 +1,93 @@
+# Undiscovered room dots
+
+Enable **Preferences → Graphics → Mapping Hints → Show undiscovered rooms**.
+The option is off by default and can be enabled while playing.
+
+A red corner dot marks a mapped room whose server ID is absent from the current
+character's complete `Room.Known` list. MUME considers rooms seen through `exits`
+known as well as rooms entered: these dots show undiscovered rooms, not a strict
+record of rooms never physically entered.
+
+Rooms without server IDs, temporary rooms, and rooms whose status is not yet
+available are not marked as undiscovered. If the existing missing-ID option is
+enabled, its yellow dots still identify rooms needing IDs. Where a room also has
+an unsaved-change dot, the red dot takes precedence while discovery hints are on.
+
+A per-server, per-character cache supplies the previous room history as soon as
+GMCP identifies the character. It is provisional: the complete `Room.Known.List`
+received in the background replaces it, including removing stale IDs. The first
+load without a valid cache may take a minute or more at the server's delivery rate.
+Enabling the checkbox shows an OK/Cancel confirmation explaining this initial
+load, with OK selected by default. Its "Don't show again." checkbox is remembered
+across restarts when confirmed with OK. Cancel leaves both settings unchanged. Opening
+the preferences or loading saved settings does not show the confirmation.
+If the option is already enabled at login or character switch, a missing or invalid
+cache also triggers this warning once for that character session, unless warnings
+have been disabled. A usable cache or an already completed server history needs no
+warning. Cancel turns off discovery hints; background cache updates may continue.
+The shared dialog follows MMapper's non-blocking message-box ownership pattern,
+without requiring Asyncify or nested event loops. Its heading is included in the
+body because macOS ignores message-box window titles. All controls are configured
+before opening the dialog.
+
+Live `Room.Known.Add` messages update the view and cache directly, even when the
+overlay is disabled. Cache reads and atomic file replacements run on one serial
+background worker per server cache. Server changes retain outstanding workers
+until idle, so switching connections does not wait for disk I/O. Writes are
+coalesced over two seconds and the final pending updates flush on normal application shutdown. An abrupt process or machine crash
+can lose recent updates; the next GMCP refresh repairs the cached view.
+Incomplete, unavailable, or malformed server lists never replace a good cache.
+Without a cache, no red dots appear until the first list finishes.
+
+Caches are compact, versioned JSON files below Qt's standard application-data
+folder in `known-rooms`. Hashed filenames and embedded identity checks separate
+characters and server host/port pairs. Invalid or incompatible files are ignored.
+Late disk reads cannot overwrite another character or a completed server refresh.
+Nothing is added to the map file. The code uses Qt's portable file APIs; durable
+browser storage across page reloads needs an IndexedDB/IDBFS integration before
+this file-based cache can be relied on in a browser. The ordinary WebAssembly
+filesystem is temporary; a successful file write alone does not make it durable.
+
+The implementation uses persistent sets for inexpensive immutable snapshots and
+reuses the existing asynchronous highlight batch and dot texture. Receiving a
+newly known room does not invalidate terrain meshes. Completed batches are keyed
+by map, display options, and discovery revision so a batch for another character
+cannot be shown. Native and WebAssembly builds use the existing rendering path;
+there are no additional dependencies or artwork files. The supported browser build
+uses Qt's multithreaded WebAssembly kit (see `Dockerfile`); this patch does not
+introduce a separate threading requirement or claim single-threaded browser support.
+
+Highlight invalidation retains an in-flight worker instead of destroying its
+`std::async` future, whose destructor can block. There is at most one highlight
+worker at a time. Changes are coalesced, and results for incompatible display
+states or characters are rejected before rendering. Waiting during application
+destruction is still permitted so workers cannot outlive their owners. The renderer accesses OpenGL only on the GUI thread;
+workers consume immutable snapshots and produce vertex data.
+
+Portability: the code uses the project's existing C++20, Qt, and Immer facilities,
+fixed-width server IDs, and the shared desktop/WebGL rendering path. No operating
+system APIs, new shader features, map-format changes, or new assets are required.
+The expanded named-color capacity is a 1 KiB uniform block. Desktop caches use
+Qt file APIs; browser persistence has the limitation described above.
+
+Protocol reference: <https://mume.org/help/gmcp_room.known>.
+
+## Validation
+
+`TestProxy` covers chunked histories, live additions during and after transfer,
+immutable snapshots, character changes during transfer, disabled/unavailable
+states, empty histories, malformed IDs and lists, invalidations, and observer
+notifications. Cache tests also cover restart, stale-ID removal by server truth,
+live additions during refresh, final-write flushing, interrupted refreshes,
+character/server separation, case-insensitive cache keys, missing-cache warnings,
+and invalid files. Standard GMCP serialization/module tests remain in the same suite.
+`TestMainWindow` checks the shared dialog's default button, cancellation, remembered
+warning preference, and automatic deletion.
+
+Manual game checks:
+
+1. Enable the option after logging in and wait for the history to finish loading.
+2. Enter or discover a marked room and verify its red dot disappears.
+3. Switch characters and verify that the old character's dots are not displayed.
+4. Toggle the option alongside the existing yellow/blue mapping hints.
+5. Disconnect or use a room without a server ID and verify no discovery claim is made.

src/CMakeLists.txt

@@ -444,6 +444,10 @@ set(mmapper_SRCS
     mpi/gotowidget.h
     mpi/findreplacewidget.cpp
     mpi/findreplacewidget.h
+    observer/KnownRoomsCache.cpp
+    observer/KnownRoomsCache.h
+    observer/KnownRooms.cpp
+    observer/KnownRooms.h
     observer/gameobserver.cpp
     observer/gameobserver.h
     opengl/Font.cpp
@@ -557,6 +561,8 @@ set(mmapper_SRCS
     preferences/clientpage.h
     preferences/generalpage.cpp
     preferences/generalpage.h
+    preferences/UndiscoveredRoomsDialog.cpp
+    preferences/UndiscoveredRoomsDialog.h
     preferences/graphicspage.cpp
     preferences/graphicspage.h
     preferences/grouppage.cpp

src/configuration/configuration.cpp

@@ -296,6 +296,8 @@ ConstString KEY_SERVER_NAME = "Server name";
 ConstString KEY_SHOW_HIDDEN_EXIT_FLAGS = "Show hidden exit flags";
 ConstString KEY_SHOW_NOTES = "Show notes";
 ConstString KEY_SHOW_UNSAVED_CHANGES = "Show unsaved changes";
+ConstString KEY_SHOW_UNDISCOVERED_ROOMS = "Show undiscovered rooms";
+ConstString KEY_SHOW_UNDISCOVERED_ROOMS_WARNING = "Show undiscovered rooms warning";
 ConstString KEY_SHOW_MISSING_MAP_ID = "Show missing map id";
 ConstString KEY_TAB_COMPLETION_DICTIONARY_SIZE = "Tab completion dictionary size";
 ConstString KEY_THEME = "Theme";
@@ -693,6 +695,8 @@ void Configuration::CanvasSettings::read(const QSettings &conf)
     mapFontPointSize.clamp(6, 72);
     showMissingMapId.set(conf.value(KEY_SHOW_MISSING_MAP_ID, true).toBool());
     showUnsavedChanges.set(conf.value(KEY_SHOW_UNSAVED_CHANGES, true).toBool());
+    showUndiscoveredRooms.set(conf.value(KEY_SHOW_UNDISCOVERED_ROOMS, false).toBool());
+    showUndiscoveredRoomsWarning.set(conf.value(KEY_SHOW_UNDISCOVERED_ROOMS_WARNING, true).toBool());
     showUnmappedExits.set(conf.value(KEY_DRAW_NOT_MAPPED_EXITS, true).toBool());
     drawUpperLayersTextured = conf.value(KEY_DRAW_UPPER_LAYERS_TEXTURED, false).toBool();
     drawDoorNames = conf.value(KEY_DRAW_DOOR_NAMES, true).toBool();
@@ -906,6 +910,8 @@ void Configuration::CanvasSettings::write(QSettings &conf) const
     conf.setValue(KEY_MAP_FONT_SIZE, mapFontPointSize.get());
     conf.setValue(KEY_SHOW_MISSING_MAP_ID, showMissingMapId.get());
     conf.setValue(KEY_SHOW_UNSAVED_CHANGES, showUnsavedChanges.get());
+    conf.setValue(KEY_SHOW_UNDISCOVERED_ROOMS, showUndiscoveredRooms.get());
+    conf.setValue(KEY_SHOW_UNDISCOVERED_ROOMS_WARNING, showUndiscoveredRoomsWarning.get());
     conf.setValue(KEY_DRAW_NOT_MAPPED_EXITS, showUnmappedExits.get());
     conf.setValue(KEY_DRAW_UPPER_LAYERS_TEXTURED, drawUpperLayersTextured);
     conf.setValue(KEY_DRAW_DOOR_NAMES, drawDoorNames);
@@ -1090,6 +1096,7 @@ void Configuration::NamedColorOptions::resetToDefaults()
     BACKGROUND = background;
     CONNECTION_NORMAL = Colors::white;
     HIGHLIGHT_UNSAVED = Colors::cyan;
+    HIGHLIGHT_UNDISCOVERED = Colors::red;
     HIGHLIGHT_TEMPORARY = Colors::red;
     HIGHLIGHT_NEEDS_SERVER_ID = Colors::yellow;
     INFOMARK_COMMENT = Colors::gray75;

src/configuration/configuration.h

@@ -181,6 +181,8 @@ class NODISCARD Configuration final
         NamedConfig<bool> trilinearFiltering{"TRILINEAR_FILTERING", true};
         NamedConfig<bool> showMissingMapId{"SHOW_MISSING_MAPID", false};
         NamedConfig<bool> showUnsavedChanges{"SHOW_UNSAVED_CHANGES", false};
+        NamedConfig<bool> showUndiscoveredRooms{"SHOW_UNDISCOVERED_ROOMS", false};
+        NamedConfig<bool> showUndiscoveredRoomsWarning{"SHOW_UNDISCOVERED_ROOMS_WARNING", true};
         NamedConfig<bool> showUnmappedExits{"SHOW_UNMAPPED_EXITS", false};
         NamedConfig<QString> mapFontFamily{"MAP_FONT_FAMILY", QStringLiteral("Cantarell")};
         NamedConfig<int> mapFontPointSize{"MAP_FONT_POINT_SIZE", platformPointSize(11)};

src/display/mapcanvas.cpp

@@ -1372,7 +1372,7 @@ void MapCanvas::layerChanged()
 void MapCanvas::forceUpdateMeshes()
 {
     m_batches.resetExistingMeshesAndIgnorePendingRemesh();
-    m_diff.resetExistingMeshesAndIgnorePendingRemesh();
+    m_diff.clearMeshes();
     m_frameManager.requestUpdate();
 }
 

src/display/mapcanvas.h

@@ -10,6 +10,7 @@
 #include "../global/Signal2.h"
 #include "../map/PromptFlags.h"
 #include "../mapdata/roomselection.h"
+#include "../observer/KnownRooms.h"
 #include "../opengl/Font.h"
 #include "../opengl/FontFormatFlags.h"
 #include "../opengl/OpenGL.h"
@@ -157,24 +158,50 @@ class NODISCARD_QOBJECT MapCanvas final : public QObject,
             }
         };
 
+        struct NODISCARD Options final
+        {
+            bool missingId = false;
+            bool changed = false;
+            bool undiscovered = false;
+            uint64_t knownRevision = 0;
+            uint64_t knownGeneration = 0;
+            bool operator==(const Options &) const = default;
+
+            // Live additions can retain the previous batch for a frame. Invalidated
+            // histories and other characters must never retain their old red dots.
+            NODISCARD bool sameVisibility(const Options &other) const
+            {
+                auto copy = *this;
+                copy.knownRevision = other.knownRevision;
+                return copy == other;
+            }
+        };
+
         struct NODISCARD HighlightDiff final
         {
             Map saved;
             Map current;
             MaybeDataOrMesh highlights;
+            Options options;
         };
 
         std::optional<std::future<HighlightDiff>> futureHighlight;
         std::optional<HighlightDiff> highlight;
 
-        NODISCARD bool isUpToDate(const Map &saved, const Map &current) const;
+        NODISCARD bool isUpToDate(const Map &saved,
+                                  const Map &current,
+                                  const Options &options) const;
         NODISCARD bool hasRelatedDiff(const Map &save) const;
         void cancelUpdates(const Map &saved);
-        void maybeAsyncUpdate(const Map &saved, const Map &current);
+        void maybeAsyncUpdate(const Map &saved,
+                              const Map &current,
+                              const KnownRooms::Snapshot &known,
+                              const Options &options);
 
-        void resetExistingMeshesAndIgnorePendingRemesh()
+        void clearMeshes()
         {
-            futureHighlight.reset();
+            // Destroying a future from std::async can wait for its worker. Keep
+            // that worker alive; its immutable result is checked before rendering.
             highlight.reset();
         }
     };

src/display/mapcanvas_gl.cpp

@@ -352,6 +352,12 @@ bool MapCanvas::hostInitializeGL()
         programs.early_init();
     }
 
+    m_observer.sig2_knownRoomsChanged.connect(m_lifetime,
+                                              [this]() { m_frameManager.requestUpdate(); });
+    setConfig().canvas.showUndiscoveredRooms.registerChangeCallback(m_lifetime, [this]() {
+        m_frameManager.requestUpdate();
+    });
+
     setConfig().canvas.showUnsavedChanges.registerChangeCallback(m_lifetime, [this]() {
         if (getConfig().canvas.showUnsavedChanges.get() && m_diff.highlight.has_value()
             && m_diff.highlight->highlights.empty()) {
@@ -719,10 +725,12 @@ void MapCanvas::actuallyPaintGL()
     }
 }
 
-NODISCARD bool MapCanvas::Diff::isUpToDate(const Map &saved, const Map &current) const
+NODISCARD bool MapCanvas::Diff::isUpToDate(const Map &saved,
+                                           const Map &current,
+                                           const Options &options) const
 {
     return highlight && highlight->saved.isSamePointer(saved)
-           && highlight->current.isSamePointer(current);
+           && highlight->current.isSamePointer(current) && highlight->options == options;
 }
 
 // this differs from isUpToDate in that it allows display of a diff based on the current saved map,
@@ -734,21 +742,25 @@ NODISCARD bool MapCanvas::Diff::hasRelatedDiff(const Map &saved) const
 
 void MapCanvas::Diff::cancelUpdates(const Map &saved)
 {
-    futureHighlight.reset();
+    // Keep an in-flight job rather than joining its worker on the GUI thread.
+    // hasRelatedDiff() and the options check reject obsolete results, and the
+    // next update coalesces all intervening changes into a fresh snapshot.
     if (highlight) {
         if (!hasRelatedDiff(saved)) {
             highlight.reset();
         }
     }
 }
 
-void MapCanvas::Diff::maybeAsyncUpdate(const Map &saved, const Map &current)
+void MapCanvas::Diff::maybeAsyncUpdate(const Map &saved,
+                                       const Map &current,
+                                       const KnownRooms::Snapshot &known,
+                                       const Options &options)
 {
     auto &diff = *this;
 
-    // Pending takes precedence. This also usually guarantees at most one pending update at a time,
-    // but calling resetExistingMeshesAndIgnorePendingRemesh() could result in more than one diff
-    // mesh thread executing concurrently, where the old one will be ignored.
+    // At most one highlight worker runs at a time, including across invalidation.
+    // Only release its future after it is ready, so invalidation never waits for it.
     if (diff.futureHighlight) {
         constexpr auto immediate = std::chrono::milliseconds(0);
         if (diff.futureHighlight->wait_for(immediate) != std::future_status::timeout) {
@@ -763,24 +775,25 @@ void MapCanvas::Diff::maybeAsyncUpdate(const Map &saved, const Map &current)
     }
 
     // no change necessary
-    if (isUpToDate(saved, current)) {
+    if (isUpToDate(saved, current, options)) {
         return;
     }
 
-    const auto &config = getConfig();
-    const auto &canvas = config.canvas;
-    const bool showNeedsServerId = canvas.showMissingMapId.get();
-    const bool showChanged = canvas.showUnsavedChanges.get();
+    const bool showNeedsServerId = options.missingId;
+    const bool showChanged = options.changed;
+    const bool showUndiscovered = options.undiscovered;
 
     diff.futureHighlight = std::async(
         std::launch::async,
-        [saved, current, showNeedsServerId, showChanged]() -> Diff::HighlightDiff {
+        [saved, current, known, options, showNeedsServerId, showChanged, showUndiscovered]()
+            -> Diff::HighlightDiff {
             DECL_TIMER(t2,
                        "[async] actuallyPaintGL: highlight changes, temporary, and needs update");
 
             auto getHighlights =
-                [&saved, &current, showChanged, showNeedsServerId]() -> Diff::MaybeDataOrMesh {
-                if (!showChanged && !showNeedsServerId) {
+                [&saved, &current, &known, showChanged, showNeedsServerId, showUndiscovered]()
+                -> Diff::MaybeDataOrMesh {
+                if (!showChanged && !showNeedsServerId && !showUndiscovered) {
                     return Diff::MaybeDataOrMesh{};
                 }
 
@@ -815,13 +828,25 @@ void MapCanvas::Diff::maybeAsyncUpdate(const Map &saved, const Map &current)
                                             });
                 }
 
+                // Red takes precedence over the existing mapping dots in the same corner.
+                // Missing IDs and temporary rooms cannot establish discovery status.
+                if (showUndiscovered) {
+                    current.getRooms().for_each([&current, &known, &drawQuad](auto id) {
+                        if (const auto room = current.getRoomHandle(id)) {
+                            if (!room.isTemporary() && known.isUndiscovered(room.getServerId())) {
+                                drawQuad(room.getRaw(), NamedColorEnum::HIGHLIGHT_UNDISCOVERED);
+                            }
+                        }
+                    });
+                }
+
                 if (highlights.empty()) {
                     return Diff::MaybeDataOrMesh{};
                 }
                 return Diff::MaybeDataOrMesh{std::move(highlights)};
             };
 
-            return Diff::HighlightDiff{saved, current, getHighlights()};
+            return Diff::HighlightDiff{saved, current, getHighlights(), options};
         });
 }
 
@@ -831,8 +856,20 @@ void MapCanvas::paintDifferences()
     const auto &saved = m_data.getSavedMap();
     const auto &current = m_data.getCurrentMap();
 
-    diff.maybeAsyncUpdate(saved, current);
-    if (!diff.hasRelatedDiff(saved)) {
+    const auto &known = m_observer.getKnownRooms();
+    const auto &canvas = getConfig().canvas;
+    const bool undiscovered = canvas.showUndiscoveredRooms.get() && known.complete;
+    const Diff::Options options{canvas.showMissingMapId.get(),
+                                canvas.showUnsavedChanges.get(),
+                                undiscovered,
+                                undiscovered ? known.revision : 0,
+                                undiscovered ? known.generation : 0};
+    diff.maybeAsyncUpdate(saved, current, known, options);
+    if (diff.futureHighlight || !diff.isUpToDate(saved, current, options)) {
+        // Finish or replace the pending batch even when the map is otherwise idle.
+        m_frameManager.requestUpdate();
+    }
+    if (!diff.hasRelatedDiff(saved) || !diff.highlight->options.sameVisibility(options)) {
         return;
     }
 

src/global/JsonDoc.h

@@ -26,7 +26,10 @@ class NODISCARD JsonDoc final
     explicit JsonDoc(const TaggedStringUtf8<Tag> &json)
         : m_doc{QJsonDocument::fromJson(json.toQByteArray())}
         , m_int{std::invoke([this, &json]() -> OptJsonInt {
-            if (m_doc.isArray() || m_doc.isObject() || json.isEmpty())
+            if (m_doc.isArray() || m_doc.isObject() || json.isEmpty()
+                || json.toQByteArray().trimmed() == "null"
+                || json.toQByteArray().trimmed() == "false"
+                || json.toQByteArray().trimmed() == "true")
                 return std::nullopt;
 
             // Qt's Json parser doesn't support integers at the top level of a document