Add various debug functions

HTTP Server disabled by default

Changelog.md

@@ -1,7 +1,7 @@
-# v2.4.0
+# v3.0.0
 
 - CHANGED entire plugin Lua implementation (rewrite)
-- CHANGED moved *almost all* functions into MP.\*
+- CHANGED moved *almost all* Lua functions into MP.\*
 - CHANGED console to use a custom language (type `help`, `list`, or `status`!)
 - CHANGED all files of a Lua plugin to share a Lua state (no more state-per-file)
 - ADDED many new Lua API functions, which can be found at <https://wiki.beammp.com/en/Scripting/functions>
@@ -14,7 +14,7 @@
 - ADDED dumping tables with `print()` (try it with `print(MP)`)
 - ADDED `MP.GetOSName()`, `MP.CreateTimer()`, `MP.GetLuaMemoryUsage()` and many more (see <https://wiki.beammp.com/en/Scripting/functions>)
 - ADDED `MP.Settings` table to make usage of `MP.Set()` easier
-- ADDED `FS.*` table with common filesystem operations
+- ADDED `FS.*` table with common filesystem operations (do `print(FS)` to see them!)
 - FIXED i/o thread spin when stdout is /dev/null on linux
 - FIXED removed extra whitespace infront of onChatMessage message
 

README.md

@@ -6,6 +6,8 @@
 This is the server for the multiplayer mod **[BeamMP](https://beammp.com/)** for the game [BeamNG.drive](https://www.beamng.com/).
 The server is the point throug which all clients communicate. You can write lua mods for the server, detailed instructions on the [BeamMP Wiki](https://wiki.beammp.com).
 
+**For Linux, you __need__ the runtime dependencies, listed below under "prerequisites".**
+
 ## Support + Contact
 
 Feel free to ask any questions via the following channels:
@@ -48,7 +50,7 @@ You can find precompiled binaries under [Releases](https://github.com/BeamMP/Bea
 
 ## Build Instructions
 
-**__Do not compile from `master`. Always build from a release tag, i.e. `tags/v2.0`!__**
+**__Do not compile from `master`. Always build from a release tag, i.e. `tags/v2.3.3`!__**
 
 Currently only linux and windows are supported (generally). See [Releases](https://github.com/BeamMP/BeamMP-Server/releases/) for official binary releases. On systems to which we do not provide binaries (so anything but windows), you are allowed to compile the program and use it. Other restrictions, such as not being allowed to distribute those binaries, still apply (see [copyright notice](#copyright)).
 
@@ -58,38 +60,47 @@ Currently only linux and windows are supported (generally). See [Releases](https
 
 Please use the prepackaged binaries in [Releases](https://github.com/BeamMP/BeamMP-Server/releases/).
 
-Dependencies for windows can be installed with `vcpkg`, in which case the current dependencies are the `x64-windows-static` versions of `lua`, `zlib`, `rapidjson`, and `openssl`.
+Dependencies for **windows** can be installed with `vcpkg`.
+These are:
+```
+lua
+zlib
+rapidjson
+openssl
+websocketpp
+curl
+```
 
-#### Linux / \*nix
+#### Linux
 
 These package names are in the debian / ubuntu style. Feel free to PR your own guide for a different distro.
 
-- `git`
+Runtime dependencies for **linux** are (debian/ubuntu):
-- `make`
+```
-- `cmake`
+libz-dev
-- `g++`
+rapidjson-dev
-- `libcurl4-openssl-dev` or similar (search for `libcurl` and look for one with `-dev`)
+liblua5.3
-  
+libssl-dev
-  Must support ISO C++17. If your distro's `g++` doesn't support C++17, chances are that it has a `g++-8` or `g++-10` package that does. If this is the case. you just need to run CMake with `-DCMAKE_CXX_COMPILER=g++-10` (replace `g++-10` with your compiler's name).
+libwebsocketpp-dev
-- `liblua5.3-dev` 
+libcurl4-openssl-dev
-  
-  Any 5.x version should work, but 5.3 is what we officially use. Any other version might break in the future.
-  You can also use any version of `libluajit`, but the same applies regarding the version.
-- `libz-dev`
-- `rapidjson-dev`
-- `libopenssl-dev` or `libssl-dev`
-
-In the end you should end up with a command something like this:
-
-```sh
-sudo apt install git make cmake g++-10 liblua5.3 libz-dev rapidjson-dev libopenssl-dev
 ```
 
-In the end you should end up with a command something like this:
+Build-time dependencies for **linux** are:
-
-```sh
-sudo apt install git make cmake g++-10 liblua5.3 libz-dev rapidjson-dev libopenssl-dev
 ```
+git
+make
+cmake
+g++
+```
+
+For other distributions (e.g. Arch) you want to find packages for:
+- libz
+- rapidjson
+- lua5.3
+- ssl / openssl
+- websocketpp
+- curl (with ssl support)
+- \+ the build time dependencies from above
 
 ### How to build
 
@@ -98,7 +109,7 @@ On windows, use git-bash for these commands. On Linux, these should work in your
 1. Make sure you have all [prerequisites](#prerequisites) installed
 2. Clone the repository in a location of your choice with `git clone --recurse-submodules https://github.com/BeamMP/BeamMP-Server`. 
 3. Ensure that all submodules are initialized by running `git submodule update --init --recursive`. Then change into the cloned directory by running `cd BeamMP-Server`.
-4. Checkout the branch of the release you want to compile (`master` is often unstable), for example `git checkout tags/v1.20` for version 1.20. You can find the latest version [here](https://github.com/BeamMP/BeamMP-Server/tags).
+4. Checkout the branch of the release you want to compile (`master` is often unstable), for example `git checkout tags/v2.3.3` for version 2.3.3. You can find the latest version [here](https://github.com/BeamMP/BeamMP-Server/tags).
 5. Run `cmake . -DCMAKE_BUILD_TYPE=Release` (with `.`)
 6. Run `make`
 7. You will now have a `BeamMP-Server` file in your directory, which is executable with `./BeamMP-Server` (`.\BeamMP-Server.exe` for windows). Follow the (windows or linux, doesnt matter) instructions on the [wiki](https://wiki.beammp.com/en/home/Server_Mod) for further setup after installation (which we just did), such as port-forwarding and getting a key to actually run the server.

include/Common.h

@@ -42,7 +42,7 @@ public:
         std::string Key {};
         std::string SSLKeyPath { "./.ssl/HttpServer/key.pem" };
         std::string SSLCertPath { "./.ssl/HttpServer/cert.pem" };
-        bool HTTPServerEnabled { true };
+        bool HTTPServerEnabled { false };
         int MaxPlayers { 10 };
         bool Private { true };
         int MaxCars { 1 };
@@ -114,7 +114,7 @@ private:
     static inline std::mutex mShutdownHandlersMutex {};
     static inline std::deque<TShutdownHandler> mShutdownHandlers {};
 
-    static inline Version mVersion { 2, 4, 0 };
+    static inline Version mVersion { 3, 0, 0 };
 };
 
 std::string ThreadName(bool DebugModeOverride = false);

include/TConsole.h

@@ -21,6 +21,7 @@ private:
     void RunAsCommand(const std::string& cmd, bool IgnoreNotACommand = false);
     void ChangeToLuaConsole(const std::string& LuaStateId);
     void ChangeToRegularConsole();
+    void HandleLuaInternalCommand(const std::string& cmd);
 
     void Command_Lua(const std::string& cmd);
     void Command_Help(const std::string& cmd);

include/TLuaEngine.h

@@ -32,8 +32,8 @@ static constexpr size_t TLuaArgTypes_Bool = 3;
 class TLuaPlugin;
 
 struct TLuaResult {
-    std::atomic_bool Ready;
+    bool Ready;
-    std::atomic_bool Error;
+    bool Error;
     std::string ErrorMessage;
     sol::object Result { sol::lua_nil };
     TLuaStateId StateId;
@@ -89,6 +89,7 @@ public:
         std::unique_lock Lock(mResultsToCheckMutex);
         return mResultsToCheck.size();
     }
+    
     size_t GetLuaStateCount() {
         std::unique_lock Lock(mLuaStatesMutex);
         return mLuaStates.size();
@@ -133,10 +134,12 @@ public:
         }
 
         std::vector<std::shared_ptr<TLuaResult>> Results;
+        std::vector<TLuaArgTypes> Arguments { TLuaArgTypes { std::forward<ArgsT>(Args) }... };
+
         for (const auto& Event : mLuaEvents.at(EventName)) {
             for (const auto& Function : Event.second) {
                 if (Event.first != IgnoreId) {
-                    Results.push_back(EnqueueFunctionCall(Event.first, Function, { TLuaArgTypes { std::forward<ArgsT>(Args) }... }));
+                    Results.push_back(EnqueueFunctionCall(Event.first, Function, Arguments));
                 }
             }
         }
@@ -150,6 +153,12 @@ public:
 
     static constexpr const char* BeamMPFnNotFoundError = "BEAMMP_FN_NOT_FOUND";
 
+    // Debugging functions (slow)
+    std::unordered_map<std::string /*event name */, std::vector<std::string> /* handlers */> Debug_GetEventsForState(TLuaStateId StateId);
+    std::queue<std::pair<TLuaChunk, std::shared_ptr<TLuaResult>>> Debug_GetStateExecuteQueueForState(TLuaStateId StateId);
+    std::queue<std::tuple<std::string, std::shared_ptr<TLuaResult>, std::vector<TLuaArgTypes>>> Debug_GetStateFunctionQueueForState(TLuaStateId StateId);
+    std::vector<TLuaResult> Debug_GetResultsToCheckForState(TLuaStateId StateId);
+
 private:
     void CollectAndInitPlugins();
     void InitializePlugin(const fs::path& Folder, const TLuaPluginConfig& Config);
@@ -168,6 +177,10 @@ private:
         void operator()() override;
         sol::state_view State() { return sol::state_view(mState); }
 
+        // Debug functions, slow
+        std::queue<std::pair<TLuaChunk, std::shared_ptr<TLuaResult>>> Debug_GetStateExecuteQueue();
+        std::queue<std::tuple<std::string, std::shared_ptr<TLuaResult>, std::vector<TLuaArgTypes>>> Debug_GetStateFunctionQueue();
+
     private:
         sol::table Lua_TriggerGlobalEvent(const std::string& EventName, sol::variadic_args EventArgs);
         sol::table Lua_TriggerLocalEvent(const std::string& EventName, sol::variadic_args EventArgs);

src/TConsole.cpp

@@ -107,10 +107,10 @@ void TConsole::ChangeToLuaConsole(const std::string& LuaStateId) {
         mStateId = LuaStateId;
         mIsLuaConsole = true;
         if (mStateId != mDefaultStateId) {
-            Application::Console().WriteRaw("Entered Lua console for state '" + mStateId + "'. To exit, type `exit()`");
+            Application::Console().WriteRaw("Attached to Lua state '" + mStateId + "'. For help, type `:help`. To detach, type `:detach`");
             mCommandline.set_prompt("lua @" + LuaStateId + "> ");
         } else {
-            Application::Console().WriteRaw("Entered Lua console. To exit, type `exit()`");
+            Application::Console().WriteRaw("Attached to Lua. For help, type `:help`. To detach, type `:detach`");
             mCommandline.set_prompt("lua> ");
         }
         mCachedRegularHistory = mCommandline.history();
@@ -122,9 +122,9 @@ void TConsole::ChangeToRegularConsole() {
     if (mIsLuaConsole) {
         mIsLuaConsole = false;
         if (mStateId != mDefaultStateId) {
-            Application::Console().WriteRaw("Left Lua console for state '" + mStateId + "'.");
+            Application::Console().WriteRaw("Detached from Lua state '" + mStateId + "'.");
         } else {
-            Application::Console().WriteRaw("Left Lua console.");
+            Application::Console().WriteRaw("Detached from Lua.");
         }
         mCachedLuaHistory = mCommandline.history();
         mCommandline.set_history(mCachedRegularHistory);
@@ -364,6 +364,58 @@ void TConsole::RunAsCommand(const std::string& cmd, bool IgnoreNotACommand) {
     }
 }
 
+void TConsole::HandleLuaInternalCommand(const std::string& cmd) {
+    if (cmd == "detach") {
+        ChangeToRegularConsole();
+    } else if (cmd == "queued") {
+        auto QueuedFunctions = LuaAPI::MP::Engine->Debug_GetStateFunctionQueueForState(mStateId);
+        Application::Console().WriteRaw("Pending functions in State '" + mStateId + "'");
+        std::unordered_map<std::string, size_t> FunctionsCount;
+        std::vector<std::string> FunctionsInOrder;
+        while (!QueuedFunctions.empty()) {
+            auto Tuple = QueuedFunctions.front();
+            QueuedFunctions.pop();
+            FunctionsInOrder.push_back(std::get<0>(Tuple));
+            FunctionsCount[std::get<0>(Tuple)] += 1;
+        }
+        std::set<std::string> Uniques;
+        for (const auto& Function : FunctionsInOrder) {
+            if (Uniques.count(Function) == 0) {
+                Uniques.insert(Function);
+                if (FunctionsCount.at(Function) > 1) {
+                    Application::Console().WriteRaw("    " + Function + " (" + std::to_string(FunctionsCount.at(Function)) + "x)");
+                } else {
+                    Application::Console().WriteRaw("    " + Function);
+                }
+            }
+        }
+        Application::Console().WriteRaw("Executed functions waiting to be checked in State '" + mStateId + "'");
+        for (const auto& Function : LuaAPI::MP::Engine->Debug_GetResultsToCheckForState(mStateId)) {
+            Application::Console().WriteRaw("    '" + Function.Function + "' (Ready? " + (Function.Ready ? "Yes" : "No") + ", Error? " + (Function.Error ? "Yes: '" + Function.ErrorMessage + "'" : "No") + ")");
+        }
+    } else if (cmd == "events") {
+        auto Events = LuaAPI::MP::Engine->Debug_GetEventsForState(mStateId);
+        Application::Console().WriteRaw("Registered Events + Handlers for State '" + mStateId + "'");
+        for (const auto& EventHandlerPair : Events) {
+            Application::Console().WriteRaw("    Event '" + EventHandlerPair.first + "'");
+            for (const auto& Handler : EventHandlerPair.second) {
+                Application::Console().WriteRaw("        " + Handler);
+            }
+        }
+    } else if (cmd == "help") {
+        Application::Console().WriteRaw(R"(BeamMP Lua Debugger
+    All commands must be prefixed with a `:`. Non-prefixed commands are interpreted as Lua.
+
+Commands
+    :detach       detaches (exits) from this Lua console
+    :help         displays this help
+    :events       shows a list of currently registered events
+    :queued       shows a list of all pending and queued functions)");
+    } else {
+        beammp_error("internal command '" + cmd + "' is not known");
+    }
+}
+
 TConsole::TConsole() {
     mCommandline.enable_history();
     mCommandline.set_history_limit(20);
@@ -381,8 +433,8 @@ TConsole::TConsole() {
             if (mIsLuaConsole) {
                 if (!mLuaEngine) {
                     beammp_info("Lua not started yet, please try again in a second");
-                } else if (cmd == "exit()") {
+                } else if (!cmd.empty() && cmd.at(0) == ':') {
-                    ChangeToRegularConsole();
+                    HandleLuaInternalCommand(cmd.substr(1));
                 } else {
                     auto Future = mLuaEngine->EnqueueScript(mStateId, { std::make_shared<std::string>(cmd), "", "" });
                     while (!Future->Ready) {

src/TLuaEngine.cpp

@@ -149,6 +149,50 @@ TLuaStateId TLuaEngine::GetStateIDForPlugin(const fs::path& PluginPath) {
     return "";
 }
 
+std::unordered_map<std::string /*event name */, std::vector<std::string> /* handlers */> TLuaEngine::Debug_GetEventsForState(TLuaStateId StateId) {
+    std::unordered_map<std::string, std::vector<std::string>> Result;
+    std::unique_lock Lock(mLuaEventsMutex);
+    for (const auto& EventNameToEventMap : mLuaEvents) {
+        for (const auto& IdSetOfHandlersPair : EventNameToEventMap.second) {
+            if (IdSetOfHandlersPair.first == StateId) {
+                for (const auto& Handler : IdSetOfHandlersPair.second) {
+                    Result[EventNameToEventMap.first].push_back(Handler);
+                }
+            }
+        }
+    }
+    return Result;
+}
+
+std::queue<std::pair<TLuaChunk, std::shared_ptr<TLuaResult>>> TLuaEngine::Debug_GetStateExecuteQueueForState(TLuaStateId StateId) {
+    std::queue<std::pair<TLuaChunk, std::shared_ptr<TLuaResult>>> Result;
+    std::unique_lock Lock(mLuaStatesMutex);
+    Result = mLuaStates.at(StateId)->Debug_GetStateExecuteQueue();
+    return Result;
+}
+
+std::queue<std::tuple<std::string, std::shared_ptr<TLuaResult>, std::vector<TLuaArgTypes>>> TLuaEngine::Debug_GetStateFunctionQueueForState(TLuaStateId StateId) {
+    std::queue<std::tuple<std::string, std::shared_ptr<TLuaResult>, std::vector<TLuaArgTypes>>> Result;
+    std::unique_lock Lock(mLuaStatesMutex);
+    Result = mLuaStates.at(StateId)->Debug_GetStateFunctionQueue();
+    return Result;
+}
+
+std::vector<TLuaResult> TLuaEngine::Debug_GetResultsToCheckForState(TLuaStateId StateId) {
+    std::unique_lock Lock(mResultsToCheckMutex);
+    auto ResultsToCheckCopy = mResultsToCheck;
+    Lock.unlock();
+    std::vector<TLuaResult> Result;
+    while (!ResultsToCheckCopy.empty()) {
+        auto ResultToCheck = std::move(ResultsToCheckCopy.front());
+        ResultsToCheckCopy.pop();
+        if (ResultToCheck->StateId == StateId) {
+            Result.push_back(*ResultToCheck);
+        }
+    }
+    return Result;
+}
+
 void TLuaEngine::WaitForAll(std::vector<std::shared_ptr<TLuaResult>>& Results, const std::optional<std::chrono::high_resolution_clock::duration>& Max) {
     for (const auto& Result : Results) {
         bool Cancelled = false;
@@ -671,6 +715,16 @@ void TLuaEngine::StateThreadData::operator()() {
     }
 }
 
+std::queue<std::pair<TLuaChunk, std::shared_ptr<TLuaResult>>> TLuaEngine::StateThreadData::Debug_GetStateExecuteQueue() {
+    std::unique_lock Lock(mStateExecuteQueueMutex);
+    return mStateExecuteQueue;
+}
+
+std::queue<std::tuple<std::string, std::shared_ptr<TLuaResult>, std::vector<TLuaArgTypes>>> TLuaEngine::StateThreadData::Debug_GetStateFunctionQueue() {
+    std::unique_lock Lock(mStateFunctionQueueMutex);
+    return mStateFunctionQueue;
+}
+
 void TLuaEngine::CreateEventTimer(const std::string& EventName, TLuaStateId StateId, size_t IntervalMS) {
     std::unique_lock Lock(mTimedEventsMutex);
     TimedEvent Event {