Version 0.3.0

- Use the SDL2 framework wholesale instead of manually extracting dylib and headers
- Run install_name_tool on the remaining dylibs so they use the correct load path
- Add generate-dmg.sh for creating packaged Mac builds

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,31 @@
+---
+name: Bug report
+about: Create a report to help us address broken functionality
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**Steps to reproduce**
+Any special steps that are required for the bug to appear.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem. If the issue is related to video glitching or poor quality, please include screenshots.
+
+**Moonlight settings (please complete the following information)**
+- Have any settings been adjusted from defaults?
+- If so, which settings have been changed?
+- Does the problem still occur after reverting settings back to default?
+
+**Client PC details (please complete the following information)**
+ - OS: [e.g. Windows 10 1803]
+ - Moonlight Version: [e.g. v0.2.0]
+
+**Logs (please attach)**
+- On Windows, `Moonlight-###.log` files can be found in `%TEMP%`. Simply type that into the File Explorer path field to navigate there.
+- On macOS, `Moonlight-###.log` files can be found in `/tmp`. In Finder, press Cmd+Shift+G, then type `/tmp` to navigate there.
+- On Linux, logs print to the terminal when running via `flatpak run com.moonlight_stream.Moonlight`
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/no-response.yml

@@ -0,0 +1,8 @@
+# ProBot No Response (https://probot.github.io/apps/no-response/)
+
+daysUntilClose: 7
+responseRequiredLabel: 'need more info'
+closeComment: >
+  This issue has been automatically closed because there was no response to a
+  request for more information from the issue opener. Please leave a comment or
+  open a new issue if you have additional information related to this issue.

.github/stale.yml

@@ -0,0 +1,14 @@
+# ProBot Stale (https://probot.github.io/apps/stale/)
+
+daysUntilStale: 90
+daysUntilClose: 7
+exemptLabels:
+  - accepted
+  - bug
+  - enhancement
+  - meta
+staleLabel: stale
+markComment: >
+  This issue has been automatically marked as stale because it has not had
+  recent activity. It will be closed if no further activity occurs.
+closeComment: false

.gitmodules

@@ -1,9 +1,6 @@
 [submodule "moonlight-common-c/moonlight-common-c"]
 	path = moonlight-common-c/moonlight-common-c
 	url = https://github.com/moonlight-stream/moonlight-common-c.git
-[submodule "opus/opus"]
-	path = opus/opus
-	url = https://github.com/xiph/opus.git
 [submodule "qmdnsengine/qmdnsengine"]
 	path = qmdnsengine/qmdnsengine
 	url = https://github.com/cgutman/qmdnsengine.git

.travis.yml

@@ -0,0 +1,49 @@
+language: cpp
+
+git:
+  depth: 1
+
+matrix:
+  include:
+    - os: osx
+      compiler: clang
+    - os: linux
+      sudo: required
+      dist: xenial
+      compiler: gcc
+      addons:
+        apt:
+          sources:
+            - sourceline: 'ppa:beineri/opt-qt596-xenial'
+            - sourceline: 'ppa:jonathonf/ffmpeg-4'
+          packages:
+            - qt59base
+            - qt59quickcontrols2
+            - qt59svg
+            - libesd0-dev
+            - libgl-dev
+            - libegl1-mesa-dev
+            - libxkbcommon-dev
+            - wayland-protocols
+            - libopus-dev
+            - libavcodec-dev
+            - libavutil-dev
+            - libva-dev
+            - libvdpau-dev
+
+install:
+    - '[ "$TRAVIS_OS_NAME" != osx ] || brew install qt5'
+    - '[ "$TRAVIS_OS_NAME" != osx ] || brew link --force qt5'
+    - if [[ "$TRAVIS_OS_NAME" = "linux" ]]; then export SDL2_VER=2.0.8 && wget https://www.libsdl.org/release/SDL2-$SDL2_VER.tar.gz && tar -xvf SDL2-$SDL2_VER.tar.gz && cd SDL2-$SDL2_VER && ./configure && make && sudo make install && cd ..; fi
+
+before_script:
+    - '[ "$TRAVIS_OS_NAME" != "linux" ] || source /opt/qt59/bin/qt59-env.sh'
+    - qmake -v
+    - '[[ "$TRAVIS_OS_NAME" != linux || "$CXX" != clang++ ]] || export QMAKESPEC=linux-clang'
+    - '[[ "$TRAVIS_OS_NAME" != linux || "$CXX" != g++     ]] || export QMAKESPEC=linux-g++'
+    - '[ "$TRAVIS_OS_NAME" != osx ]                          || export QMAKESPEC=macx-clang'
+
+script:
+    - qmake moonlight-qt.pro -spec $QMAKESPEC
+    - make debug
+    - make release

README.md

@@ -1,11 +1,30 @@
 # Moonlight PC
 
-[Moonlight PC](http://moonlight-stream.com) is an open source implementation of NVIDIA's GameStream, as used by the NVIDIA Shield, but built to run on Windows, Mac, and Linux.
+[Moonlight PC](http://moonlight-stream.com) is an open source implementation of NVIDIA's GameStream, as used by the NVIDIA Shield, but built to run on Windows, Mac, and Linux. This client is the successor to [Moonlight Chrome](https://github.com/moonlight-stream/moonlight-chrome) for streaming on PC.
 
 Moonlight also has mobile versions for [Android](https://github.com/moonlight-stream/moonlight-android) and  [iOS](https://github.com/moonlight-stream/moonlight-ios).
 
+See the [releases page](https://github.com/moonlight-stream/moonlight-qt/releases) for download links for Windows and Mac. For Linux, we're on [Flathub](https://flathub.org/apps/details/com.moonlight_stream.Moonlight).
+
+You can follow development on our [Discord server](https://discord.gg/6ERtzFY).
+
+ [![Windows AppVeyor Status](https://ci.appveyor.com/api/projects/status/glj5cxqwy2w3bglv/branch/master?svg=true)](https://ci.appveyor.com/project/cgutman/moonlight-qt/branch/master)
+ [![Mac and Linux Travis CI Status](https://travis-ci.org/moonlight-stream/moonlight-qt.svg?branch=master)](https://travis-ci.org/moonlight-stream/moonlight-qt)
+
+## Features
+ - Hardware accelerated video decoding on Windows, Mac, and Linux
+ - Supports streaming at up to 120 FPS on high refresh rate monitors
+ - Supports streaming at 720p, 1080p, 1440p, or 4K
+ - 5.1 surround sound audio
+ - HEVC support for better image quality at reduced bandwidth
+ - Keyboard and mouse support
+ - Gamepad support with SDL gamepad mappings
+
 ## Building
-1. Install the latest Qt SDK (and optionally, the Qt Creator IDE) from https://www.qt.io/download (select MSVC toolchain on Windows)
+1. Install the latest Qt SDK (and optionally, the Qt Creator IDE) from https://www.qt.io/download
+    - On Windows, install Visual Studio 2017 and select MSVC 2017 toolchain during Qt installation
+    - On Mac, install the latest version of XCode
+    - On Linux, install your distro equivalents of: `openssl-devel qt5-devel SDL2-devel ffmpeg-devel qt5-qtquickcontrols2-devel libva-devel libvdpau-devel`
 2. Run `git submodule update --init --recursive` from within `moonlight-qt/`
 3. Open the project in Qt Creator or build from qmake on the command line
 

app/Info.plist

@@ -5,22 +5,28 @@
 	<key>CFBundleExecutable</key>
 	<string>Moonlight</string>
 	<key>CFBundleGetInfoString</key>
-	<string>Created by Qt/QMake</string>
+	<string>Stream games from your NVIDIA GameStream-enabled PC</string>
 	<key>CFBundleIconFile</key>
-	<string>macos</string>
+	<string>moonlight</string>
 	<key>CFBundleIdentifier</key>
-	<string>com.yourcompany.moonlight-qt</string>
+	<string>com.moonlight-stream.Moonlight</string>
+	<key>NSHighResolutionCapable</key>
+	<true/>
 	<key>CFBundlePackageType</key>
 	<string>APPL</string>
 	<key>CFBundleSignature</key>
 	<string>????</string>
 	<key>LSMinimumSystemVersion</key>
-	<string>10.10</string>
+	<string>10.11</string>
-	<key>NOTE</key>
-	<string>This file was generated by Qt/QMake.</string>
 	<key>NSPrincipalClass</key>
 	<string>NSApplication</string>
 	<key>NSSupportsAutomaticGraphicsSwitching</key>
 	<true/>
+	<key>CFBundleVersion</key>
+	<string>0.3.0</string>
+	<key>CFBundleShortVersionString</key>
+	<string>0.3.0</string>
+	<key>CFBundleDisplayName</key>
+	<string>Moonlight</string>
 </dict>
 </plist>

app/Moonlight.exe.manifest

@@ -0,0 +1,31 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<assembly xmlns="urn:schemas-microsoft-com:asm.v1" xmlns:asmv3="urn:schemas-microsoft-com:asm.v3" manifestVersion="1.0">
+    <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
+        <security>
+            <requestedPrivileges>
+                <requestedExecutionLevel level="asInvoker" uiAccess="false" />
+            </requestedPrivileges>
+        </security>
+    </trustInfo>
+    <description>Moonlight Game Streaming</description>
+    <asmv3:application>
+        <asmv3:windowsSettings>
+            <dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true/pm</dpiAware>
+            <dpiAwareness xmlns="http://schemas.microsoft.com/SMI/2016/WindowsSettings">permonitor</dpiAwareness>
+        </asmv3:windowsSettings>
+    </asmv3:application>
+    <compatibility xmlns="urn:schemas-microsoft-com:compatibility.v1">
+        <application>
+            <!-- Windows 10 -->
+            <supportedOS Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
+            <!-- Windows 8.1 -->
+            <supportedOS Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>
+            <!-- Windows 8 -->
+            <supportedOS Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
+            <!-- Windows 7 -->
+            <supportedOS Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
+            <!-- Windows Vista -->
+            <supportedOS Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
+        </application> 
+    </compatibility>
+</assembly>

app/app.pro

@@ -8,6 +8,9 @@ unix:!macx {
     TARGET = Moonlight
 }
 
+# Support debug and release builds from command line for CI
+CONFIG += debug_and_release
+
 TEMPLATE = app
 
 # The following define makes your compiler emit warnings if you use
@@ -31,33 +34,42 @@ win32 {
         LIBS += -L$$PWD/../libs/windows/lib/x64
     }
 
-    LIBS += ws2_32.lib winmm.lib dxva2.lib ole32.lib
+    LIBS += ws2_32.lib winmm.lib dxva2.lib ole32.lib gdi32.lib user32.lib d3d9.lib dwmapi.lib
 }
 macx {
-    INCLUDEPATH += $$PWD/../libs/mac/include
+    INCLUDEPATH += $$PWD/../libs/mac/include $$PWD/../libs/mac/Frameworks/SDL2.framework/Versions/A/Headers
-    LIBS += -L$$PWD/../libs/mac/lib
+    LIBS += -L$$PWD/../libs/mac/lib -F$$PWD/../libs/mac/Frameworks
 }
 
 unix:!macx {
     CONFIG += link_pkgconfig
-    PKGCONFIG += openssl sdl2
+    PKGCONFIG += openssl sdl2 opus
-    LIBS += -ldl
 
     packagesExist(libavcodec) {
         PKGCONFIG += libavcodec libavutil
         CONFIG += ffmpeg
 
         packagesExist(libva) {
+            packagesExist(libva-x11) {
+                CONFIG += libva-x11
+            }
+            packagesExist(libva-wayland) {
+                CONFIG += libva-wayland
+            }
             CONFIG += libva
         }
+
+        packagesExist(vdpau) {
+            CONFIG += libvdpau
+        }
     }
 }
 win32 {
-    LIBS += -llibssl -llibcrypto -lSDL2 -lavcodec -lavutil
+    LIBS += -llibssl -llibcrypto -lSDL2 -lavcodec -lavutil -lopus
     CONFIG += ffmpeg
 }
 macx {
-    LIBS += -lssl -lcrypto -lSDL2 -lavcodec.58 -lavutil.56
+    LIBS += -lssl -lcrypto -lavcodec.58 -lavutil.56 -lopus -framework SDL2
     LIBS += -lobjc -framework VideoToolbox -framework AVFoundation -framework CoreVideo -framework CoreGraphics -framework CoreMedia -framework AppKit
     CONFIG += ffmpeg
 }
@@ -74,7 +86,10 @@ SOURCES += \
     streaming/session.cpp \
     streaming/audio.cpp \
     gui/computermodel.cpp \
-    gui/appmodel.cpp
+    gui/appmodel.cpp \
+    streaming/streamutils.cpp \
+    backend/autoupdatechecker.cpp \
+    path.cpp
 
 HEADERS += \
     utils.h \
@@ -88,7 +103,10 @@ HEADERS += \
     streaming/session.hpp \
     gui/computermodel.h \
     gui/appmodel.h \
-    streaming/video/decoder.h
+    streaming/video/decoder.h \
+    streaming/streamutils.h \
+    backend/autoupdatechecker.h \
+    path.h
 
 # Platform-specific renderers and decoders
 ffmpeg {
@@ -97,18 +115,44 @@ ffmpeg {
     DEFINES += HAVE_FFMPEG
     SOURCES += \
         streaming/video/ffmpeg.cpp \
-        streaming/video/ffmpeg-renderers/sdl.cpp
+        streaming/video/ffmpeg-renderers/sdl.cpp \
+        streaming/video/ffmpeg-renderers/pacer/pacer.cpp \
+        streaming/video/ffmpeg-renderers/pacer/nullthreadedvsyncsource.cpp
+
+
     HEADERS += \
         streaming/video/ffmpeg.h \
-        streaming/video/ffmpeg-renderers/renderer.h
+        streaming/video/ffmpeg-renderers/renderer.h \
+        streaming/video/ffmpeg-renderers/pacer/pacer.h \
+        streaming/video/ffmpeg-renderers/pacer/nullthreadedvsyncsource.h
 }
 libva {
     message(VAAPI renderer selected)
 
+    PKGCONFIG += libva
     DEFINES += HAVE_LIBVA
     SOURCES += streaming/video/ffmpeg-renderers/vaapi.cpp
     HEADERS += streaming/video/ffmpeg-renderers/vaapi.h
 }
+libva-x11 {
+    message(VAAPI X11 support enabled)
+
+    PKGCONFIG += libva-x11
+    DEFINES += HAVE_LIBVA_X11
+}
+libva-wayland {
+    message(VAAPI Wayland support enabled)
+
+    PKGCONFIG += libva-wayland
+    DEFINES += HAVE_LIBVA_WAYLAND
+}
+libvdpau {
+    message(VDPAU renderer selected)
+
+    DEFINES += HAVE_LIBVDPAU
+    SOURCES += streaming/video/ffmpeg-renderers/vdpau.cpp
+    HEADERS += streaming/video/ffmpeg-renderers/vdpau.h
+}
 config_SLVideo {
     message(SLVideo decoder/renderer selected)
 
@@ -120,14 +164,24 @@ config_SLVideo {
 win32 {
     message(DXVA2 renderer selected)
 
-    SOURCES += streaming/video/ffmpeg-renderers/dxva2.cpp
+    SOURCES += \
-    HEADERS += streaming/video/ffmpeg-renderers/dxva2.h
+        streaming/video/ffmpeg-renderers/dxva2.cpp \
+        streaming/video/ffmpeg-renderers/pacer/dxvsyncsource.cpp
+
+    HEADERS += \
+        streaming/video/ffmpeg-renderers/dxva2.h \
+        streaming/video/ffmpeg-renderers/pacer/dxvsyncsource.h
 }
 macx {
     message(VideoToolbox renderer selected)
 
-    SOURCES += streaming/video/ffmpeg-renderers/vt.mm
+    SOURCES += \
-    HEADERS += streaming/video/ffmpeg-renderers/vt.h
+        streaming/video/ffmpeg-renderers/vt.mm \
+        streaming/video/ffmpeg-renderers/pacer/displaylinkvsyncsource.cpp
+
+    HEADERS += \
+        streaming/video/ffmpeg-renderers/vt.h \
+        streaming/video/ffmpeg-renderers/pacer/displaylinkvsyncsource.h
 }
 
 RESOURCES += \
@@ -147,13 +201,6 @@ else:unix: LIBS += -L$$OUT_PWD/../moonlight-common-c/ -lmoonlight-common-c
 INCLUDEPATH += $$PWD/../moonlight-common-c/moonlight-common-c/src
 DEPENDPATH += $$PWD/../moonlight-common-c/moonlight-common-c/src
 
-win32:CONFIG(release, debug|release): LIBS += -L$$OUT_PWD/../opus/release/ -lopus
-else:win32:CONFIG(debug, debug|release): LIBS += -L$$OUT_PWD/../opus/debug/ -lopus
-else:unix: LIBS += -L$$OUT_PWD/../opus/ -lopus
-
-INCLUDEPATH += $$PWD/../opus/opus/include
-DEPENDPATH += $$PWD/../opus/opus/include
-
 win32:CONFIG(release, debug|release): LIBS += -L$$OUT_PWD/../qmdnsengine/release/ -lqmdnsengine
 else:win32:CONFIG(debug, debug|release): LIBS += -L$$OUT_PWD/../qmdnsengine/debug/ -lqmdnsengine
 else:unix: LIBS += -L$$OUT_PWD/../qmdnsengine/ -lqmdnsengine
@@ -182,16 +229,18 @@ unix:!macx: {
 
     INSTALLS += target desktop icons appdata
 }
-
 win32 {
     RC_ICONS = moonlight.ico
+    CONFIG -= embed_manifest_exe
+    QMAKE_LFLAGS += /MANIFEST:embed /MANIFESTINPUT:$${PWD}/Moonlight.exe.manifest
 }
-
 macx {
     QMAKE_INFO_PLIST = $$PWD/Info.plist
-    APP_QML_FILES.files = res/macos.icns
+
-    APP_QML_FILES.path = Contents/Resources
+    APP_BUNDLE_RESOURCES.files = moonlight.icns
-    QMAKE_BUNDLE_DATA += APP_QML_FILES
+    APP_BUNDLE_RESOURCES.path = Contents/Resources
+    QMAKE_BUNDLE_DATA += APP_BUNDLE_RESOURCES
 }
 
-VERSION = 0.0.3
+VERSION = 0.3.0
+DEFINES += VERSION_STR=\\\"0.3.0\\\"

app/backend/autoupdatechecker.cpp

@@ -0,0 +1,142 @@
+#include "autoupdatechecker.h"
+
+#include <QNetworkReply>
+#include <QJsonDocument>
+#include <QJsonArray>
+#include <QJsonObject>
+
+AutoUpdateChecker::AutoUpdateChecker(QObject *parent) :
+    QObject(parent),
+    m_Nam(this)
+{
+    // Never communicate over HTTP
+    m_Nam.setStrictTransportSecurityEnabled(true);
+
+    connect(&m_Nam, SIGNAL(finished(QNetworkReply*)),
+            this, SLOT(handleUpdateCheckRequestFinished(QNetworkReply*)));
+
+    QString currentVersion(VERSION_STR);
+    qDebug() << "Current Moonlight version:" << currentVersion;
+    parseStringToVersionQuad(currentVersion, m_CurrentVersionQuad);
+
+    // Should at least have a 1.0-style version number
+    Q_ASSERT(m_CurrentVersionQuad.count() > 1);
+}
+
+void AutoUpdateChecker::start()
+{
+#if defined(Q_OS_WIN32) || defined(Q_OS_DARWIN) // Only run update checker on platforms without auto-update
+    // We'll get a callback when this is finished
+    QUrl url("https://moonlight-stream.com/updates/qt.json");
+    m_Nam.get(QNetworkRequest(url));
+#endif
+}
+
+void AutoUpdateChecker::parseStringToVersionQuad(QString& string, QVector<int>& version)
+{
+    QStringList list = string.split('.');
+    for (const QString& component : list) {
+        version.append(component.toInt());
+    }
+}
+
+void AutoUpdateChecker::handleUpdateCheckRequestFinished(QNetworkReply* reply)
+{
+    Q_ASSERT(reply->isFinished());
+
+    if (reply->error() == QNetworkReply::NoError) {
+        QTextStream stream(reply);
+        stream.setCodec("UTF-8");
+
+        // Read all data and queue the reply for deletion
+        QString jsonString = stream.readAll();
+        reply->deleteLater();
+
+        QJsonParseError error;
+        QJsonDocument jsonDoc = QJsonDocument::fromJson(jsonString.toUtf8(), &error);
+        if (jsonDoc.isNull()) {
+            qWarning() << "Update manifest malformed:" << error.errorString();
+            return;
+        }
+
+        QJsonArray array = jsonDoc.array();
+        if (array.isEmpty()) {
+            qWarning() << "Update manifest doesn't contain an array";
+            return;
+        }
+
+        for (QJsonValueRef updateEntry : array) {
+            if (updateEntry.isObject()) {
+                QJsonObject updateObj = updateEntry.toObject();
+                if (!updateObj.contains("platform") ||
+                        !updateObj.contains("arch") ||
+                        !updateObj.contains("version") ||
+                        !updateObj.contains("browser_url")) {
+                    qWarning() << "Update manifest entry missing vital field";
+                    continue;
+                }
+
+                if (!updateObj["platform"].isString() ||
+                        !updateObj["arch"].isString() ||
+                        !updateObj["version"].isString() ||
+                        !updateObj["browser_url"].isString()) {
+                    qWarning() << "Update manifest entry has unexpected vital field type";
+                    continue;
+                }
+
+                if (updateObj["arch"] == QSysInfo::buildCpuArchitecture() &&
+                        updateObj["platform"] == QSysInfo::productType()) {
+                    qDebug() << "Found update manifest match for current platform";
+
+                    QString latestVersion = updateObj["version"].toString();
+                    qDebug() << "Latest version of Moonlight for this platform is:" << latestVersion;
+
+                    QVector<int> latestVersionQuad;
+
+                    parseStringToVersionQuad(latestVersion, latestVersionQuad);
+
+                    for (int i = 0;; i++) {
+                        int latestVer = 0;
+                        int currentVer = 0;
+
+                        // Treat missing decimal places as 0
+                        if (i < latestVersionQuad.count()) {
+                            latestVer = latestVersionQuad[i];
+                        }
+                        if (i < m_CurrentVersionQuad.count()) {
+                            currentVer = m_CurrentVersionQuad[i];
+                        }
+                        if (i >= latestVersionQuad.count() && i >= m_CurrentVersionQuad.count()) {
+                            break;
+                        }
+
+                        if (currentVer < latestVer) {
+                            qDebug() << "Update available";
+
+                            emit onUpdateAvailable(updateObj["browser_url"].toString());
+                            return;
+                        }
+                        else if (currentVer > latestVer) {
+                            qDebug() << "Update manifest version lower than current version";
+                            return;
+                        }
+                    }
+
+                    qDebug() << "Update manifest version equal to current version";
+
+                    return;
+                }
+            }
+            else {
+                qWarning() << "Update manifest contained unrecognized entry:" << updateEntry.toString();
+            }
+        }
+
+        qWarning() << "No entry in update manifest found for current platform: "
+                   << QSysInfo::buildCpuArchitecture() << QSysInfo::productType();
+    }
+    else {
+        qWarning() << "Update checking failed with error: " << reply->error();
+        reply->deleteLater();
+    }
+}

app/backend/autoupdatechecker.h

@@ -0,0 +1,25 @@
+#pragma once
+
+#include <QObject>
+#include <QNetworkAccessManager>
+
+class AutoUpdateChecker : public QObject
+{
+    Q_OBJECT
+public:
+    explicit AutoUpdateChecker(QObject *parent = nullptr);
+
+    Q_INVOKABLE void start();
+
+signals:
+    void onUpdateAvailable(QString url);
+
+private slots:
+    void handleUpdateCheckRequestFinished(QNetworkReply* reply);
+
+private:
+    void parseStringToVersionQuad(QString& string, QVector<int>& version);
+
+    QVector<int> m_CurrentVersionQuad;
+    QNetworkAccessManager m_Nam;
+};

app/backend/boxartmanager.cpp

@@ -1,13 +1,12 @@
 #include "boxartmanager.h"
+#include "../path.h"
 
-#include <QStandardPaths>
 #include <QImageReader>
 #include <QImageWriter>
 
 BoxArtManager::BoxArtManager(QObject *parent) :
     QObject(parent),
-    m_BoxArtDir(
+    m_BoxArtDir(Path::getBoxArtCacheDir())
-        QStandardPaths::writableLocation(QStandardPaths::CacheLocation) + "/boxart")
 {
     if (!m_BoxArtDir.exists()) {
         m_BoxArtDir.mkpath(".");

app/backend/computermanager.cpp

@@ -51,6 +51,8 @@ NvComputer::NvComputer(QSettings& settings)
     this->appVersion = nullptr;
     this->maxLumaPixelsHEVC = 0;
     this->serverCodecModeSupport = 0;
+    this->pendingQuit = false;
+    this->gpuModel = nullptr;
 }
 
 void
@@ -133,8 +135,10 @@ NvComputer::NvComputer(QString address, QString serverInfo)
     this->currentGameId = NvHTTP::getCurrentGame(serverInfo);
     this->appVersion = NvHTTP::getXmlString(serverInfo, "appversion");
     this->gfeVersion = NvHTTP::getXmlString(serverInfo, "GfeVersion");
+    this->gpuModel = NvHTTP::getXmlString(serverInfo, "gputype");
     this->activeAddress = address;
     this->state = NvComputer::CS_ONLINE;
+    this->pendingQuit = false;
 }
 
 bool NvComputer::wake()
@@ -264,6 +268,7 @@ bool NvComputer::update(NvComputer& that)
     ASSIGN_IF_CHANGED(gfeVersion);
     ASSIGN_IF_CHANGED(appVersion);
     ASSIGN_IF_CHANGED(maxLumaPixelsHEVC);
+    ASSIGN_IF_CHANGED(gpuModel);
     ASSIGN_IF_CHANGED_AND_NONEMPTY(appList);
     ASSIGN_IF_CHANGED_AND_NONEMPTY(displayModes);
     return changed;
@@ -446,6 +451,15 @@ void ComputerManager::pairHost(NvComputer* computer, QString pin)
     QThreadPool::globalInstance()->start(pairing);
 }
 
+void ComputerManager::quitRunningApp(NvComputer* computer)
+{
+    QWriteLocker lock(&computer->lock);
+    computer->pendingQuit = true;
+
+    PendingQuitTask* quit = new PendingQuitTask(this, computer);
+    QThreadPool::globalInstance()->start(quit);
+}
+
 void ComputerManager::stopPollingAsync()
 {
     QWriteLocker lock(&m_Lock);
@@ -497,6 +511,11 @@ ComputerManager::handleComputerStateChanged(NvComputer* computer)
 {
     emit computerStateChanged(computer);
 
+    if (computer->pendingQuit && computer->currentGameId == 0) {
+        computer->pendingQuit = false;
+        emit quitAppCompleted(QVariant());
+    }
+
     // Save updated hosts to QSettings
     saveHosts();
 }

app/backend/computermanager.h

@@ -16,10 +16,14 @@
 class NvComputer
 {
     friend class PcMonitorThread;
+    friend class ComputerManager;
+    friend class PendingQuitTask;
 
 private:
     void sortAppList();
 
+    bool pendingQuit;
+
 public:
     explicit NvComputer(QString address, QString serverInfo);
 
@@ -61,6 +65,7 @@ public:
     QVector<NvDisplayMode> displayModes;
     int maxLumaPixelsHEVC;
     int serverCodecModeSupport;
+    QString gpuModel;
 
     // Persisted traits
     QString localAddress;
@@ -148,32 +153,27 @@ private:
         int pollsSinceLastAppListFetch = POLLS_PER_APPLIST_FETCH;
         while (!isInterruptionRequested()) {
             bool stateChanged = false;
-            for (int i = 0; i < TRIES_BEFORE_OFFLINING; i++) {
+            bool online = false;
+            for (int i = 0; i < TRIES_BEFORE_OFFLINING && !online; i++) {
                 for (auto& address : m_Computer->uniqueAddresses()) {
                     if (isInterruptionRequested()) {
                         return;
                     }
 
                     if (tryPollComputer(address, stateChanged)) {
+                        online = true;
                         break;
                     }
                 }
-
-                // No need to continue retrying if we're online
-                if (m_Computer->state == NvComputer::CS_ONLINE) {
-                    break;
-                }
             }
 
             // Check if we failed after all retry attempts
             // Note: we don't need to acquire the read lock here,
             // because we're on the writing thread.
-            if (m_Computer->state != NvComputer::CS_ONLINE) {
+            if (!online && m_Computer->state != NvComputer::CS_OFFLINE) {
-                if (m_Computer->state != NvComputer::CS_OFFLINE) {
                 m_Computer->state = NvComputer::CS_OFFLINE;
                 stateChanged = true;
             }
-            }
 
             // Grab the applist if it's empty or it's been long enough that we need to refresh
             pollsSinceLastAppListFetch++;
@@ -259,6 +259,8 @@ public:
 
     void pairHost(NvComputer* computer, QString pin);
 
+    void quitRunningApp(NvComputer* computer);
+
     QVector<NvComputer*> getComputers();
 
     // computer is deleted inside this call
@@ -271,6 +273,8 @@ signals:
 
     void computerAddCompleted(QVariant success);
 
+    void quitAppCompleted(QVariant error);
+
 private slots:
     void handleComputerStateChanged(NvComputer* computer);
 
@@ -338,6 +342,49 @@ private:
     QString m_Pin;
 };
 
+class PendingQuitTask : public QObject, public QRunnable
+{
+    Q_OBJECT
+
+public:
+    PendingQuitTask(ComputerManager* computerManager, NvComputer* computer)
+        : m_Computer(computer)
+    {
+        connect(this, &PendingQuitTask::quitAppFailed,
+                computerManager, &ComputerManager::quitAppCompleted);
+    }
+
+signals:
+    void quitAppFailed(QString error);
+
+private:
+    void run()
+    {
+        NvHTTP http(m_Computer->activeAddress);
+
+        try {
+            if (m_Computer->currentGameId != 0) {
+                http.quitApp();
+            }
+        } catch (const GfeHttpResponseException& e) {
+            {
+                QWriteLocker lock(&m_Computer->lock);
+                m_Computer->pendingQuit = false;
+            }
+            if (e.getStatusCode() == 599) {
+                // 599 is a special code we make a custom message for
+                emit quitAppFailed("The running game wasn't started by this PC. "
+                                   "You must quit the game on the host PC manually or use the device that originally started the game.");
+            }
+            else {
+                emit quitAppFailed(e.toQString());
+            }
+        }
+    }
+
+    NvComputer* m_Computer;
+};
+
 class PendingAddTask : public QObject, public QRunnable
 {
     Q_OBJECT

app/backend/nvhttp.cpp

@@ -16,6 +16,8 @@
 NvHTTP::NvHTTP(QString address) :
     m_Address(address)
 {
+    Q_ASSERT(!address.isEmpty());
+
     m_BaseUrlHttp.setScheme("http");
     m_BaseUrlHttps.setScheme("https");
     m_BaseUrlHttp.setHost(address);
@@ -67,6 +69,7 @@ NvHTTP::getServerInfo()
         serverInfo = openConnectionToString(m_BaseUrlHttps,
                                             "serverinfo",
                                             nullptr,
+                                            true,
                                             true);
         // Throws if the request failed
         verifyResponseStatus(serverInfo);
@@ -79,6 +82,7 @@ NvHTTP::getServerInfo()
             serverInfo = openConnectionToString(m_BaseUrlHttp,
                                                 "serverinfo",
                                                 nullptr,
+                                                true,
                                                 true);
             verifyResponseStatus(serverInfo);
         }
@@ -225,6 +229,7 @@ NvHTTP::getAppList()
     QString appxml = openConnectionToString(m_BaseUrlHttps,
                                             "applist",
                                             nullptr,
+                                            true,
                                             true);
     verifyResponseStatus(appxml);
 
@@ -338,9 +343,10 @@ QString
 NvHTTP::openConnectionToString(QUrl baseUrl,
                                QString command,
                                QString arguments,
-                               bool enableTimeout)
+                               bool enableTimeout,
+                               bool suppressLogging)
 {
-    QNetworkReply* reply = openConnection(baseUrl, command, arguments, enableTimeout);
+    QNetworkReply* reply = openConnection(baseUrl, command, arguments, enableTimeout, suppressLogging);
     QString ret;
 
     QTextStream stream(reply);
@@ -355,7 +361,8 @@ QNetworkReply*
 NvHTTP::openConnection(QUrl baseUrl,
                        QString command,
                        QString arguments,
-                       bool enableTimeout)
+                       bool enableTimeout,
+                       bool suppressLogging)
 {
     // Build a URL for the request
     QUrl url(baseUrl);
@@ -381,13 +388,17 @@ NvHTTP::openConnection(QUrl baseUrl,
     {
         QTimer::singleShot(REQUEST_TIMEOUT_MS, &loop, SLOT(quit()));
     }
+    if (!suppressLogging) {
         qInfo() << "Executing request:" << url.toString();
+    }
     loop.exec(QEventLoop::ExcludeUserInputEvents);
 
     // Abort the request if it timed out
     if (!reply->isFinished())
     {
+        if (!suppressLogging) {
             qWarning() << "Aborting timed out request for" << url.toString();
+        }
         reply->abort();
     }
 
@@ -398,7 +409,9 @@ NvHTTP::openConnection(QUrl baseUrl,
     // Handle error
     if (reply->error() != QNetworkReply::NoError)
     {
+        if (!suppressLogging) {
             qWarning() << command << " request failed with error " << reply->error();
+        }
         GfeHttpResponseException exception(reply->error(), reply->errorString());
         delete reply;
         throw exception;

app/backend/nvhttp.h

@@ -107,7 +107,8 @@ public:
     openConnectionToString(QUrl baseUrl,
                            QString command,
                            QString arguments,
-                           bool enableTimeout);
+                           bool enableTimeout,
+                           bool suppressLogging = false);
 
     static
     QVector<int>
@@ -143,7 +144,8 @@ private:
     openConnection(QUrl baseUrl,
                    QString command,
                    QString arguments,
-                   bool enableTimeout);
+                   bool enableTimeout,
+                   bool suppressLogging = false);
 
     QString m_Address;
     QNetworkAccessManager m_Nam;

app/deploy/linux/com.moonlight_stream.Moonlight.appdata.xml

@@ -1,20 +1,96 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <component type="desktop">
-	<id>com.moonlight_stream.Moonlight.desktop</id>
+	<id>com.moonlight_stream.Moonlight</id>
 	<metadata_license>CC0-1.0</metadata_license>
 	<project_license>GPL-3.0+</project_license>
 	<name>Moonlight</name>
 	<summary>Stream games from your NVIDIA GameStream-enabled PC</summary>
 
 	<description>
-		<p>Moonlight can stream games or other applications from a PC with an NVIDIA GeForce GTX 600-series or higher GPU and GeForce Experience installed.</p>
+		<p>Moonlight can stream games and other applications from a PC with an NVIDIA GeForce GTX 600-series or higher GPU and GeForce Experience installed.</p>
-		<p>GameStream utilizes the NVENC hardware encoder present on NVIDIA GPUs for low-latency streaming at up 4K 60 FPS. Moonlight can utilize VAAPI hardware acceleration for high-performance H.264 and HEVC video decoding with low CPU usage.</p>
+		<p>Features include:</p>
+		<ul>
+			<li>Streaming at up to 4K resolution</li>
+			<li>Support for up to 120 FPS streaming on high refresh rate monitors</li>
+			<li>Hardware accelerated video decoding with VAAPI and VDPAU support</li>
+			<li>5.1 surround sound audio support</li>
+			<li>HEVC support for better video compression efficiency</li>
+			<li>Gamepad support with SDL mapping compatibility</li>
+		</ul>
 	</description>
 
 	<url type="homepage">https://moonlight-stream.com</url>
 	<url type="bugtracker">https://github.com/moonlight-stream/moonlight-qt/issues</url>
 	<url type="help">https://github.com/moonlight-stream/moonlight-docs/wiki/Setup-Guide</url>
 
+	<screenshots>
+		<screenshot type="default">
+			<image>https://moonlight-stream.com/appstream_resources/moonlight_screenshot_1.png</image>
+		</screenshot>
+		<screenshot>
+			<image>https://moonlight-stream.com/appstream_resources/moonlight_screenshot_2.png</image>
+		</screenshot>
+	</screenshots>
+
+	<releases>
+		<release version="0.3.0" date="2018-08-21">
+			<description>
+				<p>Bugfixes:</p>
+				<ul>
+					<li>Reduced power consumption when the app is minimized</li>
+					<li>Fixed malformed asset requests being sent when the first host in the grid is offline</li>
+				</ul>
+			</description>
+		</release>
+		<release version="0.2.0" date="2018-08-12">
+			<description>
+				<p>Bugfixes:</p>
+				<ul>
+					<li>Fixed functionality of '=', 'Numpad Dot', and 'Numpad Enter' keys</li>
+					<li>Fixed uninitialized variable causing strange gamepad behavior</li>
+					<li>Added a warning for incompatible configs and decoder failures</li>
+					<li>Added a warning for Wayland performance issues</li>
+					<li>Stopped capturing the mouse by default for windowed streams</li>
+				</ul>
+			</description>
+		</release>
+		<release version="0.1.0" date="2018-08-05">
+			<description>
+				<p>New features:</p>
+				<ul>
+					<li>Allow streaming at the client's native resolution</li>
+					<li>VDPAU hardware decoding support for NVIDIA GPUs</li>
+					<li>Updated UI to replace status text with icons</li>
+					<li>Stream window is now resizable</li>
+					<li>Display warnings during game launch for configuration issues</li>
+				</ul>
+				<p>Bugfixes:</p>
+				<ul>
+					<li>Gamepad input now works even if the app doesn't have focus</li>
+					<li>Fixed streaming black screen if the host supported HEVC but the client didn't</li>
+					<li>Fixed broken video scaling causing extra stretching and off-center video</li>
+					<li>Fixed gamepad input issue causing some Y axis inputs to be reversed</li>
+				</ul>
+			</description>
+		</release>
+		<release version="0.0.5" date="2018-08-01">
+			<description>
+				<p>Changes from 0.0.4:</p>
+				<ul>
+					<li>Fixed a bug causing increased frame drops and input lag</li>
+					<li>Added support for quitting running games</li>
+					<li>Added support for waking sleeping PCs with Wake-on-LAN</li>
+					<li>Increased the FPS limit to 120 FPS</li>
+				</ul>
+			</description>
+		</release>
+		<release version="0.0.4a" date="2018-07-30">
+			<description>
+				<p>Initial pre-alpha release for Linux. Please file bugs!</p>
+			</description>
+		</release>
+	</releases>
+
 	<categories>
 		<category>Game</category>
 		<category>Qt</category>
@@ -29,33 +105,5 @@
 		<binary>moonlight</binary>
 	</provides>
 
-	<content_rating type="oars-1.1">
+	<content_rating type="oars-1.0" />
-		<content_attribute id="violence-cartoon">none</content_attribute>
-		<content_attribute id="violence-fantasy">none</content_attribute>
-		<content_attribute id="violence-realistic">none</content_attribute>
-		<content_attribute id="violence-bloodshed">none</content_attribute>
-		<content_attribute id="violence-sexual">none</content_attribute>
-		<content_attribute id="violence-desecration">none</content_attribute>
-		<content_attribute id="violence-slavery">none</content_attribute>
-		<content_attribute id="violence-worship">none</content_attribute>
-		<content_attribute id="drugs-alcohol">none</content_attribute>
-		<content_attribute id="drugs-narcotics">none</content_attribute>
-		<content_attribute id="drugs-tobacco">none</content_attribute>
-		<content_attribute id="sex-nudity">none</content_attribute>
-		<content_attribute id="sex-themes">none</content_attribute>
-		<content_attribute id="sex-homosexuality">none</content_attribute>
-		<content_attribute id="sex-prostitution">none</content_attribute>
-		<content_attribute id="sex-adultery">none</content_attribute>
-		<content_attribute id="sex-appearance">none</content_attribute>
-		<content_attribute id="language-profanity">none</content_attribute>
-		<content_attribute id="language-humor">none</content_attribute>
-		<content_attribute id="language-discrimination">none</content_attribute>
-		<content_attribute id="social-chat">none</content_attribute>
-		<content_attribute id="social-info">moderate</content_attribute>
-		<content_attribute id="social-audio">none</content_attribute>
-		<content_attribute id="social-location">none</content_attribute>
-		<content_attribute id="social-contacts">none</content_attribute>
-		<content_attribute id="money-purchasing">none</content_attribute>
-		<content_attribute id="money-gambling">none</content_attribute>
-	</content_rating>
 </component>

app/gui/AppView.qml

@@ -1,4 +1,5 @@
 import QtQuick 2.9
+import QtQuick.Dialogs 1.3
 import QtQuick.Controls 2.2
 
 import AppModel 1.0
@@ -27,14 +28,18 @@ GridView {
     // routine to run, but only if we start as invisible
     visible: false
 
+    function computerLost()
+    {
+        // Go back to the PC view on PC loss
+        stackView.pop()
+    }
+
     onVisibleChanged: {
         if (visible) {
-            // Start polling when this view is shown
+            appModel.computerLost.connect(computerLost)
-            ComputerManager.startPolling()
         }
         else {
-            // Stop polling when this view is not on top
+            appModel.computerLost.disconnect(computerLost)
-            ComputerManager.stopPollingAsync()
         }
     }
 
@@ -50,12 +55,6 @@ GridView {
     delegate: Item {
         width: 200; height: 300;
 
-        Component.onCompleted: {
-            if (model.running) {
-                appNameText.text = "<font color=\"green\">Running</font><font color=\"white\"> - </font>" + appNameText.text
-            }
-        }
-
         Image {
             id: appIcon
             anchors.horizontalCenter: parent.horizontalCenter;
@@ -66,11 +65,21 @@ GridView {
             }
         }
 
+        Image {
+            id: runningIcon
+            anchors.centerIn: appIcon
+            visible: model.running
+            source: "qrc:/res/baseline-play_circle_filled_white-48px.svg"
+            sourceSize {
+                width: 75
+                height: 75
+            }
+        }
+
         Text {
             id: appNameText
             text: model.name
             color: "white"
-
             width: parent.width
             height: 125
             anchors.top: appIcon.bottom
@@ -80,17 +89,84 @@ GridView {
             elide: Text.ElideRight
         }
 
-        MouseArea {
+        function launchOrResumeSelectedApp()
-            anchors.fill: parent
+        {
-            onClicked: {
+            var runningIndex = appModel.getRunningAppIndex()
-                // TODO: Check if a different game is running
+            if (runningIndex >= 0 && runningIndex !== index) {
+                quitAppDialog.appName = appModel.getRunningAppName()
+                quitAppDialog.segueToStream = true
+                quitAppDialog.open()
+                return
+            }
 
             var component = Qt.createComponent("StreamSegue.qml")
-                var segue = component.createObject(stackView)
+            var segue = component.createObject(stackView, {"appName": model.name, "session": appModel.createSessionForApp(index)})
-                segue.appname = model.name
-                segue.session = appModel.createSessionForApp(index)
             stackView.push(segue)
         }
+
+        MessageDialog {
+            id: quitAppDialog
+            modality:Qt.WindowModal
+            property string appName : "";
+            property bool segueToStream : false
+            text:"Are you sure you want to quit " + appName +"? Any unsaved progress will be lost."
+            standardButtons: StandardButton.Yes | StandardButton.No
+            onYes: {
+                var component = Qt.createComponent("QuitSegue.qml")
+                var params = {"appName": appName}
+                if (segueToStream) {
+                    // Store the session and app name if we're going to stream after
+                    // successfully quitting the old app.
+                    params.nextAppName = model.name
+                    params.nextSession = appModel.createSessionForApp(index)
+                }
+                else {
+                    params.nextAppName = null
+                    params.nextSession = null
+                }
+
+                stackView.push(component.createObject(stackView, params))
+
+                // Trigger the quit after pushing the quit segue on screen
+                appModel.quitRunningApp()
+            }
+        }
+
+        MouseArea {
+            anchors.fill: parent
+            acceptedButtons: Qt.LeftButton | Qt.RightButton
+            onClicked: {
+                if (mouse.button === Qt.LeftButton) {
+                    // Nothing is running or this app is running
+                    launchOrResumeSelectedApp()
+                }
+                else {
+                    // Right click
+                    appContextMenu.open()
+                }
+            }
+        }
+
+        Menu {
+            id: appContextMenu
+            MenuItem {
+                text: model.running ? "Resume Game" : "Launch Game"
+                onTriggered: {
+                    appContextMenu.close()
+                    launchOrResumeSelectedApp()
+                }
+                height: visible ? implicitHeight : 0
+            }
+            MenuItem {
+                text: "Quit Game"
+                onTriggered: {
+                    quitAppDialog.appName = appModel.getRunningAppName()
+                    quitAppDialog.segueToStream = false
+                    quitAppDialog.open()
+                }
+                visible: model.running
+                height: visible ? implicitHeight : 0
+            }
         }
     }
 

app/gui/PcView.qml

@@ -2,6 +2,7 @@ import QtQuick 2.9
 import QtQuick.Controls 2.2
 import QtQuick.Dialogs 1.3
 import QtQuick.Layouts 1.3
+import QtQuick.Window 2.2
 
 import ComputerModel 1.0
 
@@ -34,30 +35,19 @@ GridView {
         // Setup signals on CM
         ComputerManager.computerAddCompleted.connect(addComplete)
 
-        if (!prefs.hasAnyHardwareAcceleration()) {
+        if (prefs.isRunningWayland()) {
+            waylandDialog.open()
+        }
+        else if (!prefs.hasAnyHardwareAcceleration()) {
             noHwDecoderDialog.open()
         }
     }
 
-    onVisibleChanged: {
-        if (visible) {
-            // Start polling when this view is shown
-            ComputerManager.startPolling()
-        }
-        else {
-            // Stop polling when this view is not top-most
-            ComputerManager.stopPollingAsync()
-        }
-    }
-
     function pairingComplete(error)
     {
         // Close the PIN dialog
         pairDialog.close()
 
-        // Start polling again
-        ComputerManager.startPolling()
-
         // Display a failed dialog if we got an error
         if (error !== null) {
             errorDialog.text = error
@@ -81,14 +71,14 @@ GridView {
         return model
     }
 
-    model: createModel()
+    model: computerModel
 
     delegate: Item {
         width: 300; height: 300;
 
         Image {
             id: pcIcon
-            anchors.horizontalCenter: parent.horizontalCenter;
+            anchors.horizontalCenter: parent.horizontalCenter
             source: {
                 model.addPc ? "qrc:/res/ic_add_to_queue_white_48px.svg" : "qrc:/res/ic_tv_white_48px.svg"
             }
@@ -98,6 +88,30 @@ GridView {
             }
         }
 
+        Image {
+            // TODO: Tooltip
+            id: stateIcon
+            anchors.horizontalCenter: pcIcon.horizontalCenter
+            anchors.verticalCenter: pcIcon.verticalCenter
+            anchors.verticalCenterOffset: -10
+            visible: !model.addPc && !model.statusUnknown && (!model.online || !model.paired)
+            source: !model.online ? "qrc:/res/baseline-warning-24px.svg" : "qrc:/res/baseline-lock-24px.svg"
+            sourceSize {
+                width: 75
+                height: 75
+            }
+        }
+
+        BusyIndicator {
+            id: statusUnknownSpinner
+            anchors.horizontalCenter: pcIcon.horizontalCenter
+            anchors.verticalCenter: pcIcon.verticalCenter
+            anchors.verticalCenterOffset: -10
+            width: 75
+            height: 75
+            visible: !model.addPc && model.statusUnknown
+        }
+
         Text {
             id: pcNameText
             text: model.name
@@ -110,41 +124,29 @@ GridView {
             wrapMode: Text.Wrap
         }
 
-        Text {
+        Menu {
-            function getStatusText(model)
+            id: pcContextMenu
-            {
+            MenuItem {
-                if (model.online) {
+                text: "Wake PC"
-                    var text = "<font color=\"green\">Online</font>"
+                onTriggered: computerModel.wakeComputer(index)
-                    text += "<font color=\"white\"> - </font>"
+                visible: !model.addPc && !model.online && model.wakeable
-                    if (model.paired) {
+                height: visible ? implicitHeight : 0
-                        text += "<font color=\"skyblue\">Paired</font>"
             }
-                    else if (model.busy) {
+            MenuItem {
-                        text += "<font color=\"red\">Busy</font>"
+                text: "Delete PC"
-                    }
+                onTriggered: {
-                    else {
+                    deletePcDialog.pcIndex = index
-                        text += "<font color=\"red\">Not Paired</font>"
+                    // get confirmation first, actual closing is called from the dialog
-                    }
+                    deletePcDialog.open()
-                    return text
-                }
-                else {
-                    return "<font color=\"red\">Offline</font>";
                 }
             }
-
-            id: pcPairedText
-            text: getStatusText(model)
-            visible: !model.addPc
-
-            width: parent.width
-            anchors.top: pcNameText.bottom
-            font.pointSize: 24
-            horizontalAlignment: Text.AlignHCenter
         }
 
         MouseArea {
             anchors.fill: parent
+            acceptedButtons: Qt.LeftButton | Qt.RightButton;
             onClicked: {
+                if(mouse.button === Qt.LeftButton) {
                     if (model.addPc) {
                         addPcDialog.open()
                     }
@@ -152,18 +154,13 @@ GridView {
                         if (model.paired) {
                             // go to game view
                             var component = Qt.createComponent("AppView.qml")
-                        var appView = component.createObject(stackView)
+                            var appView = component.createObject(stackView, {"computerIndex": index, "objectName": model.name})
-                        appView.computerIndex = index
-                        appView.objectName = model.name
                             stackView.push(appView)
                         }
                         else {
                             if (!model.busy) {
                                 var pin = ("0000" + Math.floor(Math.random() * 10000)).slice(-4)
 
-                            // Stop polling, since pairing may make GFE unresponsive
-                            ComputerManager.stopPollingAsync()
-
                                 // Kick off pairing in the background
                                 computerModel.pairComputer(index, pin)
 
@@ -177,9 +174,15 @@ GridView {
                                 errorDialog.open()
                             }
                         }
+                    } else if(!model.online) {
+                        pcContextMenu.open()
+                    }
+
+                }
+                else { // right click
+                    if(!model.addPc) { // but only for actual PCs, not the add-pc option
+                        pcContextMenu.open()
                     }
-                else {
-                    // TODO: Wake on LAN and delete PC options
                 }
             }
         }
@@ -206,6 +209,18 @@ GridView {
         }
     }
 
+    MessageDialog {
+        id: waylandDialog
+        modality:Qt.WindowModal
+        icon: StandardIcon.Warning
+        standardButtons: StandardButton.Ok | StandardButton.Help
+        text: "Moonlight does not support hardware acceleration on Wayland. Continuing on Wayland may result in poor streaming performance. " +
+              "Please switch to an X session for optimal performance."
+        onHelp: {
+            Qt.openUrlExternally("https://github.com/moonlight-stream/moonlight-docs/wiki/Fixing-Hardware-Decoding-Problems");
+        }
+    }
+
     MessageDialog {
         id: pairDialog
         // don't allow edits to the rest of the window while open
@@ -218,6 +233,19 @@ GridView {
         }
     }
 
+    MessageDialog {
+        id: deletePcDialog
+        // don't allow edits to the rest of the window while open
+        modality:Qt.WindowModal
+        property int pcIndex : -1;
+        text:"Are you sure you want to remove this PC?"
+        standardButtons: StandardButton.Yes | StandardButton.No
+        onYes: {
+            console.log("deleting PC pairing for PC at index: " + pcIndex)
+            computerModel.deleteComputer(pcIndex);
+        }
+    }
+
     Dialog {
         id: addPcDialog
         property string label: "Enter the IP address of your GameStream PC"

app/gui/QuitSegue.qml

@@ -0,0 +1,78 @@
+import QtQuick 2.0
+import QtQuick.Controls 2.2
+import QtQuick.Dialogs 1.3
+
+import ComputerManager 1.0
+import Session 1.0
+
+Item {
+    property string appName
+    property Session nextSession : null
+    property string nextAppName : ""
+
+    property string stageText : "Quitting " + appName + "..."
+
+    anchors.fill: parent
+
+    // The StackView will trigger a visibility change when
+    // we're pushed onto it, causing our onVisibleChanged
+    // routine to run, but only if we start as invisible
+    visible: false
+
+    function quitAppCompleted(error)
+    {
+        // Display a failed dialog if we got an error
+        if (error !== undefined) {
+            errorDialog.text = error
+            errorDialog.open()
+        }
+
+        // Exit this view
+        stackView.pop()
+
+        // If we're supposed to launch another game after this, do so now
+        if (error === undefined && nextSession !== null) {
+            var component = Qt.createComponent("StreamSegue.qml")
+            var segue = component.createObject(stackView, {"appName": nextAppName, "session": nextSession})
+            stackView.push(segue)
+        }
+    }
+
+    onVisibleChanged: {
+        if (visible) {
+            // Connect the quit completion signal
+            ComputerManager.quitAppCompleted.connect(quitAppCompleted)
+        }
+        else {
+            // Disconnect the signal
+            ComputerManager.quitAppCompleted.disconnect(quitAppCompleted)
+        }
+    }
+
+    Row {
+        anchors.centerIn: parent
+        spacing: 5
+
+        BusyIndicator {
+            id: stageSpinner
+        }
+
+        Label {
+            id: stageLabel
+            height: stageSpinner.height
+            text: stageText
+            font.pointSize: 20
+            verticalAlignment: Text.AlignVCenter
+
+            wrapMode: Text.Wrap
+            color: "white"
+        }
+    }
+
+    MessageDialog {
+        id: errorDialog
+        modality:Qt.WindowModal
+        icon: StandardIcon.Critical
+        standardButtons: StandardButton.Ok
+    }
+}

app/gui/SettingsView.qml

@@ -43,7 +43,7 @@ ScrollView {
                 Label {
                     width: parent.width
                     id: resFPSdesc
-                    text: qsTr("Setting values too high for your PC may cause lag, stuttering, or errors")
+                    text: qsTr("Setting values too high for your PC may cause lag, stuttering, or errors.")
                     font.pointSize: 9
                     wrapMode: Text.Wrap
                     color: "white"
@@ -55,6 +55,41 @@ ScrollView {
                     ComboBox {
                         // ignore setting the index at first, and actually set it when the component is loaded
                         Component.onCompleted: {
+                            // Add native resolutions for all attached displays
+                            for (var displayIndex = 0;; displayIndex++) {
+                                var screenRect = prefs.getDisplayResolution(displayIndex)
+                                if (screenRect.width === 0) {
+                                    // Exceeded max count of displays
+                                    break
+                                }
+
+                                var indexToAdd = 0
+                                for (var j = 0; j < resolutionComboBox.count; j++) {
+                                    var existing_width = parseInt(resolutionListModel.get(j).video_width);
+                                    var existing_height = parseInt(resolutionListModel.get(j).video_height);
+
+                                    if (screenRect.width === existing_width && screenRect.height === existing_height) {
+                                        // Duplicate entry, skip
+                                        indexToAdd = -1
+                                        break
+                                    }
+                                    else if (screenRect.width * screenRect.height > existing_width * existing_height) {
+                                        // Candidate entrypoint after this entry
+                                        indexToAdd = j + 1
+                                    }
+                                }
+
+                                // Insert this display's resolution if it's not a duplicate
+                                if (indexToAdd >= 0) {
+                                    resolutionListModel.insert(indexToAdd,
+                                                               {
+                                                                   "text": "Native ("+screenRect.width+"x"+screenRect.height+")",
+                                                                   "video_width": ""+screenRect.width,
+                                                                   "video_height": ""+screenRect.height
+                                                               })
+                                }
+                            }
+
                             // load the saved width/height, and iterate through the ComboBox until a match is found
                             // and set it to that index.
                             var saved_width = prefs.width
@@ -63,17 +98,21 @@ ScrollView {
                             for (var i = 0; i < resolutionComboBox.count; i++) {
                                 var el_width = parseInt(resolutionListModel.get(i).video_width);
                                 var el_height = parseInt(resolutionListModel.get(i).video_height);
-                                if (saved_width === el_width && saved_height === el_height) {
+
+                                // Pick the highest value lesser or equal to the saved resolution
+                                if (saved_width * saved_height >= el_width * el_height) {
                                     currentIndex = i
                                 }
                             }
                         }
 
                         id: resolutionComboBox
-                        font.pointSize: 9
+                        width: 200
                         textRole: "text"
                         model: ListModel {
                             id: resolutionListModel
+                            // Other elements may be added at runtime
+                            // based on attached display resolution
                             ListElement {
                                 text: "720p"
                                 video_width: "1280"
@@ -121,14 +160,15 @@ ScrollView {
                             currentIndex = 0
                             for (var i = 0; i < fpsComboBox.count; i++) {
                                 var el_fps = parseInt(fpsListModel.get(i).video_fps);
-                                if (el_fps === saved_fps) {
+
+                                // Pick the highest value lesser or equal to the saved FPS
+                                if (saved_fps >= el_fps) {
                                     currentIndex = i
                                 }
                             }
                         }
 
                         id: fpsComboBox
-                        font.pointSize: 9
                         textRole: "text"
                         model: ListModel {
                             id: fpsListModel
@@ -190,6 +230,7 @@ ScrollView {
                     }
                 }
 
+                Row {
                     CheckBox {
                         id: fullScreenCheck
                         text: "<font color=\"white\">Full-screen</font>"
@@ -199,6 +240,18 @@ ScrollView {
                             prefs.fullScreen = checked
                         }
                     }
+
+                    CheckBox {
+                        id: vsyncCheck
+                        text: "<font color=\"white\">Enable V-Sync</font>"
+                        font.pointSize:  12
+                        visible: fullScreenCheck.checked
+                        checked: prefs.enableVsync
+                        onCheckedChanged: {
+                            prefs.enableVsync = checked
+                        }
+                    }
+                }
             }
         }
 
@@ -238,7 +291,6 @@ ScrollView {
 
                     id: audioComboBox
                     width: Math.min(bitrateDesc.implicitWidth, parent.width)
-                    font.pointSize: 9
                     textRole: "text"
                     model: ListModel {
                         id: audioListModel
@@ -362,7 +414,6 @@ ScrollView {
 
                     id: decoderComboBox
                     width: Math.min(bitrateDesc.implicitWidth, parent.width)
-                    font.pointSize: 9
                     textRole: "text"
                     model: ListModel {
                         id: decoderListModel
@@ -409,7 +460,6 @@ ScrollView {
 
                     id: codecComboBox
                     width: Math.min(bitrateDesc.implicitWidth, parent.width)
-                    font.pointSize: 9
                     textRole: "text"
                     model: ListModel {
                         id: codecListModel

app/gui/StreamSegue.qml

@@ -6,8 +6,8 @@ import Session 1.0
 
 Item {
     property Session session
-    property string appname
+    property string appName
-    property string stageText : "Starting " + appname + "..."
+    property string stageText : "Starting " + appName + "..."
 
     anchors.fill: parent
 
@@ -47,7 +47,13 @@ Item {
 
     function displayLaunchWarning(text)
     {
-        // TODO: toast
+        // This toast appears for 3 seconds, just shorter than how long
+        // Session will wait for it to be displayed. This gives it time
+        // to transition to invisible before continuing.
+        var toast = Qt.createQmlObject('import QtQuick.Controls 2.3; ToolTip {}', parent, '')
+        toast.text = text
+        toast.timeout = 3000
+        toast.visible = true
     }
 
     onVisibleChanged: {

app/gui/appmodel.cpp

@@ -19,6 +19,32 @@ void AppModel::initialize(ComputerManager* computerManager, int computerIndex)
     m_CurrentGameId = m_Computer->currentGameId;
 }
 
+int AppModel::getRunningAppIndex()
+{
+    if (m_CurrentGameId != 0) {
+        for (int i = 0; i < m_Apps.count(); i++) {
+            if (m_Apps[i].id == m_CurrentGameId) {
+                return i;
+            }
+        }
+    }
+
+    return -1;
+}
+
+QString AppModel::getRunningAppName()
+{
+    if (m_CurrentGameId != 0) {
+        for (int i = 0; i < m_Apps.count(); i++) {
+            if (m_Apps[i].id == m_CurrentGameId) {
+                return m_Apps[i].name;
+            }
+        }
+    }
+
+    return nullptr;
+}
+
 Session* AppModel::createSessionForApp(int appIndex)
 {
     Q_ASSERT(appIndex < m_Apps.count());
@@ -70,6 +96,11 @@ QHash<int, QByteArray> AppModel::roleNames() const
     return names;
 }
 
+void AppModel::quitRunningApp()
+{
+    m_ComputerManager->quitRunningApp(m_Computer);
+}
+
 void AppModel::handleComputerStateChanged(NvComputer* computer)
 {
     // Ignore updates for computers that aren't ours
@@ -77,6 +108,14 @@ void AppModel::handleComputerStateChanged(NvComputer* computer)
         return;
     }
 
+    // If the computer has gone offline or we've been unpaired,
+    // signal the UI so we can go back to the PC view.
+    if (m_Computer->state == NvComputer::CS_OFFLINE ||
+            m_Computer->pairState == NvComputer::PS_NOT_PAIRED) {
+        emit computerLost();
+        return;
+    }
+
     // First, process additions/removals from the app list. This
     // is required because the new game may now be running, so
     // we can't check that first.

app/gui/appmodel.h

@@ -25,6 +25,12 @@ public:
 
     Q_INVOKABLE Session* createSessionForApp(int appIndex);
 
+    Q_INVOKABLE int getRunningAppIndex();
+
+    Q_INVOKABLE QString getRunningAppName();
+
+    Q_INVOKABLE void quitRunningApp();
+
     QVariant data(const QModelIndex &index, int role) const override;
 
     int rowCount(const QModelIndex &parent) const override;
@@ -36,6 +42,9 @@ private slots:
 
     void handleBoxArtLoaded(NvComputer* computer, NvApp app, QUrl image);
 
+signals:
+    void computerLost();
+
 private:
     NvComputer* m_Computer;
     BoxArtManager m_BoxArtManager;

app/gui/computermodel.cpp

@@ -46,8 +46,12 @@ QVariant ComputerModel::data(const QModelIndex& index, int role) const
         return computer->pairState == NvComputer::PS_PAIRED;
     case BusyRole:
         return computer->currentGameId != 0;
+    case WakeableRole:
+        return !computer->macAddress.isEmpty();
     case AddPcRole:
         return false;
+    case StatusUnknownRole:
+        return computer->state == NvComputer::CS_UNKNOWN;
     default:
         return QVariant();
     }
@@ -74,6 +78,8 @@ QHash<int, QByteArray> ComputerModel::roleNames() const
     names[PairedRole] = "paired";
     names[BusyRole] = "busy";
     names[AddPcRole] = "addPc";
+    names[WakeableRole] = "wakeable";
+    names[StatusUnknownRole] = "statusUnknown";
 
     return names;
 }

app/gui/computermodel.h

@@ -13,6 +13,8 @@ class ComputerModel : public QAbstractListModel
         OnlineRole,
         PairedRole,
         BusyRole,
+        WakeableRole,
+        StatusUnknownRole,
         AddPcRole
     };
 

app/gui/main.qml

@@ -1,10 +1,16 @@
 import QtQuick 2.9
 import QtQuick.Controls 2.2
 import QtQuick.Layouts 1.3
+import QtQuick.Window 2.2
 
 import QtQuick.Controls.Material 2.1
 
+import ComputerManager 1.0
+import AutoUpdateChecker 1.0
+
 ApplicationWindow {
+    property bool pollingActive: false
+
     id: window
     visible: true
     width: 1280
@@ -19,6 +25,24 @@ ApplicationWindow {
         anchors.fill: parent
     }
 
+    onVisibilityChanged: {
+        // We don't want to just use 'active' here because that will stop polling if
+        // we lose focus, which might be overzealous for users with multiple screens
+        // where we may be clearly visible on the other display. Ideally we'll poll
+        // only if the window is visible to the user (not if obscured by other windows),
+        // but it seems difficult to do this portably.
+        var shouldPoll = visibility !== Window.Minimized && visibility !== Window.Hidden
+
+        if (shouldPoll && !pollingActive) {
+            ComputerManager.startPolling()
+            pollingActive = true
+        }
+        else if (!shouldPoll && pollingActive) {
+            ComputerManager.stopPollingAsync()
+            pollingActive = false
+        }
+    }
+
     function navigateTo(url, objectName)
     {
         var existingItem = stackView.find(function(item, index) {
@@ -66,14 +90,54 @@ ApplicationWindow {
                 Layout.fillWidth: true
             }
 
+            ToolButton {
+                property string browserUrl: ""
+
+                id: updateButton
+                icon.source: "qrc:/res/update.svg"
+                icon.color: "#0BF200"
+
+                ToolTip.delay: 1000
+                ToolTip.timeout: 3000
+                ToolTip.visible: hovered
+                ToolTip.text: "Update available for Moonlight"
+
+                // Invisible until we get a callback notifying us that
+                // an update is available
+                visible: false
+
+                onClicked: Qt.openUrlExternally(browserUrl);
+
+                Menu {
+                    x: parent.width
+                    transformOrigin: Menu.TopRight
+                }
+
+
+                function updateAvailable(url)
+                {
+                    updateButton.browserUrl = url
+                    updateButton.visible = true
+                }
+
+                Component.onCompleted: {
+                    AutoUpdateChecker.onUpdateAvailable.connect(updateAvailable)
+                    AutoUpdateChecker.start()
+                }
+            }
+
             ToolButton {
                 icon.source: "qrc:/res/question_mark.svg"
 
+                ToolTip.delay: 1000
+                ToolTip.timeout: 3000
+                ToolTip.visible: hovered
+                ToolTip.text: "Help"
+
                 // TODO need to make sure browser is brought to foreground.
                 onClicked: Qt.openUrlExternally("https://github.com/moonlight-stream/moonlight-docs/wiki/Setup-Guide");
 
                 Menu {
-                    id: helpButton
                     x: parent.width
                     transformOrigin: Menu.TopRight
                 }
@@ -83,11 +147,15 @@ ApplicationWindow {
                 // TODO: Implement gamepad mapping then unhide this button
                 visible: false
 
+                ToolTip.delay: 1000
+                ToolTip.timeout: 3000
+                ToolTip.visible: hovered
+                ToolTip.text: "Gamepad Mapper"
+
                 icon.source: "qrc:/res/ic_videogame_asset_white_48px.svg"
                 onClicked: navigateTo("qrc:/gui/GamepadMapper.qml", "Gamepad Mapping")
 
                 Menu {
-                    id: gamepadMappingMenu
                     x: parent.width
                     transformOrigin: Menu.TopRight
                 }
@@ -97,8 +165,12 @@ ApplicationWindow {
                 icon.source: "qrc:/res/settings.svg"
                 onClicked: navigateTo("qrc:/gui/SettingsView.qml", "Settings")
 
+                ToolTip.delay: 1000
+                ToolTip.timeout: 3000
+                ToolTip.visible: hovered
+                ToolTip.text: "Settings"
+
                 Menu {
-                    id: optionsMenu
                     x: parent.width
                     transformOrigin: Menu.TopRight
                 }

app/main.cpp

@@ -11,8 +11,10 @@
 #define SDL_MAIN_HANDLED
 #include <SDL.h>
 
+#include "path.h"
 #include "gui/computermodel.h"
 #include "gui/appmodel.h"
+#include "backend/autoupdatechecker.h"
 #include "streaming/session.hpp"
 #include "settings/streamingpreferences.h"
 
@@ -129,9 +131,23 @@ void qtLogToDiskHandler(QtMsgType type, const QMessageLogContext&, const QString
 
 int main(int argc, char *argv[])
 {
+    if (QFile(QDir::currentPath() + "/portable.dat").exists()) {
+        qInfo() << "Running in portable mode from:" << QDir::currentPath();
+        QSettings::setDefaultFormat(QSettings::IniFormat);
+        QSettings::setPath(QSettings::IniFormat, QSettings::UserScope, QDir::currentPath());
+        QSettings::setPath(QSettings::IniFormat, QSettings::SystemScope, QDir::currentPath());
+
+        // Initialize paths for portable mode
+        Path::initialize(true);
+    }
+    else {
+        // Initialize paths for standard installation
+        Path::initialize(false);
+    }
+
 #ifdef USE_CUSTOM_LOGGER
 #ifdef LOG_TO_FILE
-    QDir tempDir(QDir::tempPath());
+    QDir tempDir(Path::getLogDir());
     s_LoggerFile = new QFile(tempDir.filePath(QString("Moonlight-%1.log").arg(QDateTime::currentSecsSinceEpoch())));
     if (s_LoggerFile->open(QIODevice::WriteOnly)) {
         qInfo() << "Redirecting log output to " << s_LoggerFile->fileName();
@@ -172,6 +188,11 @@ int main(int argc, char *argv[])
                                               [](QQmlEngine*, QJSEngine*) -> QObject* {
                                                   return new ComputerManager();
                                               });
+    qmlRegisterSingletonType<AutoUpdateChecker>("AutoUpdateChecker", 1, 0,
+                                                "AutoUpdateChecker",
+                                                [](QQmlEngine*, QJSEngine*) -> QObject* {
+                                                    return new AutoUpdateChecker();
+                                                });
 
     QQuickStyle::setStyle("Material");
 
@@ -188,6 +209,11 @@ int main(int argc, char *argv[])
                      SDL_GetError());
     }
 
+    // Avoid the default behavior of changing the timer resolution to 1 ms.
+    // We don't want this all the time that Moonlight is open. We will set
+    // it manually when we start streaming.
+    SDL_SetHint(SDL_HINT_TIMER_RESOLUTION, "0");
+
     int err = app.exec();
 
     SDL_Quit();

app/path.cpp

@@ -0,0 +1,39 @@
+#include "path.h"
+
+#include <QtDebug>
+#include <QDir>
+#include <QStandardPaths>
+#include <QSettings>
+
+QString Path::s_LogDir;
+QString Path::s_BoxArtCacheDir;
+
+QString Path::getLogDir()
+{
+    Q_ASSERT(!s_LogDir.isEmpty());
+    return s_LogDir;
+}
+
+QString Path::getBoxArtCacheDir()
+{
+    Q_ASSERT(!s_BoxArtCacheDir.isEmpty());
+    return s_BoxArtCacheDir;
+}
+
+void Path::initialize(bool portable)
+{
+    if (portable) {
+        s_LogDir = QDir::currentPath();
+        s_BoxArtCacheDir = QDir::currentPath() + "/boxart";
+    }
+    else {
+#ifdef Q_OS_DARWIN
+        // On macOS, $TMPDIR is some random folder under /var/folders/ that nobody can
+        // easily find, so use the system's global tmp directory instead.
+        s_LogDir = "/tmp";
+#else
+        s_LogDir = QDir::tempPath();
+#endif
+        s_BoxArtCacheDir = QStandardPaths::writableLocation(QStandardPaths::CacheLocation) + "/boxart";
+    }
+}

app/path.h

@@ -0,0 +1,17 @@
+#pragma once
+
+#include <QString>
+
+class Path
+{
+public:
+    static QString getLogDir();
+
+    static QString getBoxArtCacheDir();
+
+    static void initialize(bool portable);
+
+private:
+    static QString s_LogDir;
+    static QString s_BoxArtCacheDir;
+};

app/qml.qrc

@@ -6,5 +6,6 @@
         <file>gui/SettingsView.qml</file>
         <file>gui/StreamSegue.qml</file>
         <file>gui/GamepadMapper.qml</file>
+        <file>gui/QuitSegue.qml</file>
     </qresource>
 </RCC>

app/res/baseline-lock-24px.svg

@@ -0,0 +1,4 @@
+<svg xmlns="http://www.w3.org/2000/svg" fill="#FFFFFF" width="24" height="24" viewBox="0 0 24 24">
+    <path d="M0 0h24v24H0z" fill="none"/>
+    <path d="M18 8h-1V6c0-2.76-2.24-5-5-5S7 3.24 7 6v2H6c-1.1 0-2 .9-2 2v10c0 1.1.9 2 2 2h12c1.1 0 2-.9 2-2V10c0-1.1-.9-2-2-2zm-6 9c-1.1 0-2-.9-2-2s.9-2 2-2 2 .9 2 2-.9 2-2 2zm3.1-9H8.9V6c0-1.71 1.39-3.1 3.1-3.1 1.71 0 3.1 1.39 3.1 3.1v2z"/>
+</svg>

app/res/baseline-play_circle_filled_white-48px.svg

@@ -0,0 +1,4 @@
+<svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" viewBox="0 0 48 48" fill="#fff">
+    <path d="M0 0h48v48H0z" fill="none"/>
+    <path d="M24 4C12.95 4 4 12.95 4 24s8.95 20 20 20 20-8.95 20-20S35.05 4 24 4zm-4 29V15l12 9-12 9z"/>
+</svg>

app/res/baseline-warning-24px.svg

@@ -0,0 +1,4 @@
+<svg xmlns="http://www.w3.org/2000/svg" fill="#FFFFFF" width="24" height="24" viewBox="0 0 24 24">
+    <path d="M0 0h24v24H0z" fill="none"/>
+    <path d="M1 21h22L12 2 1 21zm12-3h-2v-2h2v2zm0-4h-2v-4h2v4z"/>
+</svg>

app/res/update.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="24" height="24" viewBox="0 0 24 24">
+    <path d="M21,10.12H14.22L16.96,7.3C14.23,4.6 9.81,4.5 7.08,7.2C4.35,9.91 4.35,14.28 7.08,17C9.81,19.7 14.23,19.7 16.96,17C18.32,15.65 19,14.08 19,12.1H21C21,14.08 20.12,16.65 18.36,18.39C14.85,21.87 9.15,21.87 5.64,18.39C2.14,14.92 2.11,9.28 5.62,5.81C9.13,2.34 14.76,2.34 18.27,5.81L21,3V10.12M12.5,8V12.25L16,14.33L15.28,15.54L11,13V8H12.5Z" />
+</svg>

app/resources.qrc

@@ -5,10 +5,14 @@
         <file>res/ic_videogame_asset_white_48px.svg</file>
         <file>res/ic_tv_white_48px.svg</file>
         <file>res/ic_add_to_queue_white_48px.svg</file>
+        <file>res/baseline-lock-24px.svg</file>
+        <file>res/baseline-play_circle_filled_white-48px.svg</file>
+        <file>res/baseline-warning-24px.svg</file>
         <file>res/no_app_image.png</file>
         <file>res/settings.svg</file>
         <file>res/arrow_left.svg</file>
         <file>res/question_mark.svg</file>
         <file>res/moonlight.svg</file>
+        <file>res/update.svg</file>
     </qresource>
 </RCC>

app/settings/streamingpreferences.cpp

@@ -9,6 +9,7 @@
 #define SER_FPS "fps"
 #define SER_BITRATE "bitrate"
 #define SER_FULLSCREEN "fullscreen"
+#define SER_VSYNC "vsync"
 #define SER_GAMEOPTS "gameopts"
 #define SER_HOSTAUDIO "hostaudio"
 #define SER_MULTICONT "multicontroller"
@@ -30,6 +31,7 @@ void StreamingPreferences::reload()
     fps = settings.value(SER_FPS, 60).toInt();
     bitrateKbps = settings.value(SER_BITRATE, getDefaultBitrate(width, height, fps)).toInt();
     fullScreen = settings.value(SER_FULLSCREEN, true).toBool();
+    enableVsync = settings.value(SER_VSYNC, true).toBool();
     gameOptimizations = settings.value(SER_GAMEOPTS, true).toBool();
     playAudioOnHost = settings.value(SER_HOSTAUDIO, false).toBool();
     multiController = settings.value(SER_MULTICONT, true).toBool();
@@ -50,6 +52,7 @@ void StreamingPreferences::save()
     settings.setValue(SER_FPS, fps);
     settings.setValue(SER_BITRATE, bitrateKbps);
     settings.setValue(SER_FULLSCREEN, fullScreen);
+    settings.setValue(SER_VSYNC, enableVsync);
     settings.setValue(SER_GAMEOPTS, gameOptimizations);
     settings.setValue(SER_HOSTAUDIO, playAudioOnHost);
     settings.setValue(SER_MULTICONT, multiController);
@@ -67,6 +70,11 @@ bool StreamingPreferences::hasAnyHardwareAcceleration()
                                               1920, 1080, 60);
 }
 
+bool StreamingPreferences::isRunningWayland()
+{
+    return qgetenv("XDG_SESSION_TYPE") == QByteArray("wayland");
+}
+
 int StreamingPreferences::getMaximumStreamingFrameRate()
 {
     // Never let the maximum drop below 60 FPS
@@ -82,9 +90,9 @@ int StreamingPreferences::getMaximumStreamingFrameRate()
     for (int i = 0; i < SDL_GetNumVideoDisplays(); i++) {
         SDL_DisplayMode mode;
         if (SDL_GetCurrentDisplayMode(i, &mode) == 0) {
-            // Cap the frame rate at 90 FPS, since I can't seem to get
+            // Cap the frame rate at 120 FPS. Past this, the encoders start
-            // much more out of GFE even with the game rendering at > 300 FPS.
+            // to max out and drop frames.
-            maxFrameRate = qMax(maxFrameRate, qMin(90, mode.refresh_rate));
+            maxFrameRate = qMax(maxFrameRate, qMin(120, mode.refresh_rate));
         }
     }
 
@@ -93,15 +101,50 @@ int StreamingPreferences::getMaximumStreamingFrameRate()
     return maxFrameRate;
 }
 
+QRect StreamingPreferences::getDisplayResolution(int displayIndex)
+{
+    if (SDL_InitSubSystem(SDL_INIT_VIDEO) != 0) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "SDL_InitSubSystem(SDL_INIT_VIDEO) failed: %s",
+                     SDL_GetError());
+        return QRect();
+    }
+
+    if (displayIndex >= SDL_GetNumVideoDisplays()) {
+        SDL_QuitSubSystem(SDL_INIT_VIDEO);
+        return QRect();
+    }
+
+    SDL_DisplayMode mode;
+    int err = SDL_GetCurrentDisplayMode(displayIndex, &mode);
+    SDL_QuitSubSystem(SDL_INIT_VIDEO);
+
+    if (err == 0) {
+        return QRect(0, 0, mode.w, mode.h);
+    }
+    else {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "SDL_GetCurrentDisplayMode() failed: %s",
+                     SDL_GetError());
+        return QRect();
+    }
+}
+
 int StreamingPreferences::getDefaultBitrate(int width, int height, int fps)
 {
-    if (width * height <= 1280 * 720) {
+    // This table prefers 16:10 resolutions because they are
+    // only slightly more pixels than the 16:9 equivalents, so
+    // we don't want to bump those 16:10 resolutions up to the
+    // next 16:9 slot.
+
+    // This covers 1280x720 and 1280x800 too
+    if (width * height <= 1366 * 768) {
         return static_cast<int>(5000 * (fps / 30.0));
     }
-    else if (width * height <= 1920 * 1080) {
+    else if (width * height <= 1920 * 1200) {
         return static_cast<int>(10000 * (fps / 30.0));
     }
-    else if (width * height <= 2560 * 1440) {
+    else if (width * height <= 2560 * 1600) {
         return static_cast<int>(20000 * (fps / 30.0));
     }
     else /* if (width * height <= 3840 * 2160) */ {

app/settings/streamingpreferences.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include <QObject>
+#include <QRect>
 
 class StreamingPreferences : public QObject
 {
@@ -16,8 +17,12 @@ public:
 
     Q_INVOKABLE static bool hasAnyHardwareAcceleration();
 
+    Q_INVOKABLE static bool isRunningWayland();
+
     Q_INVOKABLE static int getMaximumStreamingFrameRate();
 
+    Q_INVOKABLE QRect getDisplayResolution(int displayIndex);
+
     void reload();
 
     enum AudioConfig
@@ -50,6 +55,7 @@ public:
     Q_PROPERTY(int fps MEMBER fps NOTIFY displayModeChanged)
     Q_PROPERTY(int bitrateKbps MEMBER bitrateKbps NOTIFY bitrateChanged)
     Q_PROPERTY(bool fullScreen MEMBER fullScreen NOTIFY fullScreenChanged)
+    Q_PROPERTY(bool enableVsync MEMBER enableVsync NOTIFY enableVsyncChanged)
     Q_PROPERTY(bool gameOptimizations MEMBER gameOptimizations NOTIFY gameOptimizationsChanged)
     Q_PROPERTY(bool playAudioOnHost MEMBER playAudioOnHost NOTIFY playAudioOnHostChanged)
     Q_PROPERTY(bool multiController MEMBER multiController NOTIFY multiControllerChanged)
@@ -63,6 +69,7 @@ public:
     int fps;
     int bitrateKbps;
     bool fullScreen;
+    bool enableVsync;
     bool gameOptimizations;
     bool playAudioOnHost;
     bool multiController;
@@ -74,6 +81,7 @@ signals:
     void displayModeChanged();
     void bitrateChanged();
     void fullScreenChanged();
+    void enableVsyncChanged();
     void gameOptimizationsChanged();
     void playAudioOnHostChanged();
     void multiControllerChanged();

app/streaming/audio.cpp

@@ -142,6 +142,10 @@ int Session::sdlAudioInit(int /* audioConfiguration */,
     s_SampleIndex = 0;
     s_PendingDrops = s_PendingHardDrops = 0;
 
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Audio stream has %d channels",
+                opusConfig->channelCount);
+
     return 0;
 }
 
@@ -178,8 +182,13 @@ void Session::sdlAudioDecodeAndPlaySample(char* sampleData, int sampleLength)
     Uint32 queuedAudio = qMax((int)SDL_GetQueuedAudioSize(s_AudioDevice) - (int)s_BaselinePendingData, 0);
     Uint32 framesQueued = queuedAudio / (SAMPLES_PER_FRAME * s_ChannelCount * sizeof(short));
 
+    // We must check this prior to the below checks to ensure we don't
+    // underflow if framesQueued - s_PendingHardDrops < 0.
+    if (framesQueued <= MIN_QUEUED_FRAMES) {
+        s_PendingDrops = s_PendingHardDrops = 0;
+    }
     // Pend enough drops to get us back to MIN_QUEUED_FRAMES
-    if (framesQueued - s_PendingHardDrops > STOP_THE_WORLD_LIMIT) {
+    else if (framesQueued - s_PendingHardDrops > STOP_THE_WORLD_LIMIT) {
         s_PendingHardDrops = framesQueued - MIN_QUEUED_FRAMES;
         SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
                     "Pending hard drop of %u audio frames",
@@ -190,10 +199,7 @@ void Session::sdlAudioDecodeAndPlaySample(char* sampleData, int sampleLength)
     }
 
     // Determine if this frame should be dropped
-    if (framesQueued <= MIN_QUEUED_FRAMES) {
+    if (s_PendingHardDrops != 0) {
-        s_PendingDrops = s_PendingHardDrops = 0;
-    }
-    else if (s_PendingHardDrops != 0) {
         // Hard drops happen all at once to forcefully
         // resync with the source.
         s_PendingHardDrops--;

app/streaming/input.cpp

@@ -2,6 +2,8 @@
 #include <SDL.h>
 #include "streaming/session.hpp"
 
+#include <QtGlobal>
+
 #define VK_0 0x30
 #define VK_A 0x41
 
@@ -23,6 +25,9 @@ const int SdlInputHandler::k_ButtonMap[] = {
 SdlInputHandler::SdlInputHandler(bool multiController)
     : m_MultiController(multiController)
 {
+    // Allow gamepad input when the app doesn't have focus
+    SDL_SetHint(SDL_HINT_JOYSTICK_ALLOW_BACKGROUND_EVENTS, "1");
+
     // We need to reinit this each time, since you only get
     // an initial set of gamepad arrival events once per init.
     SDL_assert(!SDL_WasInit(SDL_INIT_GAMECONTROLLER));
@@ -36,6 +41,10 @@ SdlInputHandler::SdlInputHandler(bool multiController)
         // Player 1 is always present in non-MC mode
         m_GamepadMask = 0x1;
     }
+    else {
+        // Otherwise, detect gamepads on the fly
+        m_GamepadMask = 0;
+    }
 
     SDL_zero(m_GamepadState);
 }
@@ -153,6 +162,7 @@ void SdlInputHandler::handleKeyEvent(SDL_KeyboardEvent* event)
             case SDL_SCANCODE_CLEAR:
                 keyCode = 0x0C;
                 break;
+            case SDL_SCANCODE_KP_ENTER: // FIXME: Is this correct?
             case SDL_SCANCODE_RETURN:
                 keyCode = 0x0D;
                 break;
@@ -230,7 +240,7 @@ void SdlInputHandler::handleKeyEvent(SDL_KeyboardEvent* event)
             case SDL_SCANCODE_KP_MINUS:
                 keyCode = 0x6D;
                 break;
-            case SDL_SCANCODE_KP_DECIMAL:
+            case SDL_SCANCODE_KP_PERIOD:
                 keyCode = 0x6E;
                 break;
             case SDL_SCANCODE_KP_DIVIDE:
@@ -284,6 +294,9 @@ void SdlInputHandler::handleKeyEvent(SDL_KeyboardEvent* event)
             case SDL_SCANCODE_SEMICOLON:
                 keyCode = 0xBA;
                 break;
+            case SDL_SCANCODE_EQUALS:
+                keyCode = 0xBB;
+                break;
             case SDL_SCANCODE_COMMA:
                 keyCode = 0xBC;
                 break;
@@ -377,8 +390,8 @@ void SdlInputHandler::handleMouseMotionEvent(SDL_MouseMotionEvent* event)
     }
 
     if (event->xrel != 0 || event->yrel != 0) {
-        LiSendMouseMoveEvent((unsigned short)event->xrel,
+        LiSendMouseMoveEvent((short)event->xrel,
-                             (unsigned short)event->yrel);
+                             (short)event->yrel);
     }
 }
 
@@ -438,13 +451,18 @@ void SdlInputHandler::handleControllerAxisEvent(SDL_ControllerAxisEvent* event)
             state->lsX = event->value;
             break;
         case SDL_CONTROLLER_AXIS_LEFTY:
-            state->lsY = -event->value;
+            // Signed values have one more negative value than
+            // positive value, so inverting the sign on -32768
+            // could actually cause the value to overflow and
+            // wrap around to be negative again. Avoid that by
+            // capping the value at 32767.
+            state->lsY = -qMax(event->value, (short)-32767);
             break;
         case SDL_CONTROLLER_AXIS_RIGHTX:
             state->rsX = event->value;
             break;
         case SDL_CONTROLLER_AXIS_RIGHTY:
-            state->rsY = -event->value;
+            state->rsY = -qMax(event->value, (short)-32767);
             break;
         case SDL_CONTROLLER_AXIS_TRIGGERLEFT:
             state->lt = (unsigned char)(event->value * 255UL / 32767);

app/streaming/session.cpp

@@ -5,6 +5,12 @@
 #include <SDL.h>
 #include "utils.h"
 
+// HACK: Need to call SetThreadExecutionState() because SDL doesn't
+#ifdef Q_OS_WIN32
+#define WIN32_LEAN_AND_MEAN
+#include <Windows.h>
+#endif
+
 #ifdef HAVE_FFMPEG
 #include "video/ffmpeg.h"
 #endif
@@ -13,10 +19,8 @@
 #include "video/sl.h"
 #endif
 
-#ifdef Q_OS_WIN32
+#if defined(Q_OS_DARWIN)
-// Using full-screen desktop allows us to avoid needing to enable V-sync
+// Using full-screen desktop allows allows Spaces to work on macOS.
-// and it also avoids some strange flickering issues on my Win7 test machine
-// with Intel HD 5500 graphics.
 #define SDL_OS_FULLSCREEN_FLAG SDL_WINDOW_FULLSCREEN_DESKTOP
 #else
 #define SDL_OS_FULLSCREEN_FLAG SDL_WINDOW_FULLSCREEN
@@ -106,7 +110,7 @@ void Session::clLogMessage(const char* format, ...)
 
 bool Session::chooseDecoder(StreamingPreferences::VideoDecoderSelection vds,
                             SDL_Window* window, int videoFormat, int width, int height,
-                            int frameRate, IVideoDecoder*& chosenDecoder)
+                            int frameRate, bool enableVsync, IVideoDecoder*& chosenDecoder)
 {
 #ifdef HAVE_SLVIDEO
     chosenDecoder = new SLVideoDecoder();
@@ -125,7 +129,7 @@ bool Session::chooseDecoder(StreamingPreferences::VideoDecoderSelection vds,
 
 #ifdef HAVE_FFMPEG
     chosenDecoder = new FFmpegVideoDecoder();
-    if (chosenDecoder->initialize(vds, window, videoFormat, width, height, frameRate)) {
+    if (chosenDecoder->initialize(vds, window, videoFormat, width, height, frameRate, enableVsync)) {
         SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
                     "FFmpeg-based video decoder chosen");
         return true;
@@ -138,6 +142,10 @@ bool Session::chooseDecoder(StreamingPreferences::VideoDecoderSelection vds,
     }
 #endif
 
+#if !defined(HAVE_FFMPEG) && !defined(HAVE_SLVIDEO)
+#error No video decoding libraries available!
+#endif
+
     // If we reach this, we didn't initialize any decoders successfully
     return false;
 }
@@ -153,6 +161,9 @@ int Session::drSetup(int videoFormat, int width, int height, int frameRate, void
     // don't have to hide and show the SDL window (which seems to
     // cause pointer hiding to break on Windows).
 
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION, "Video stream is %dx%dx%d (format 0x%x)",
+                width, height, frameRate, videoFormat);
+
     return 0;
 }
 
@@ -205,7 +216,7 @@ bool Session::isHardwareDecodeAvailable(StreamingPreferences::VideoDecoderSelect
         return false;
     }
 
-    if (!chooseDecoder(vds, window, videoFormat, width, height, frameRate, decoder)) {
+    if (!chooseDecoder(vds, window, videoFormat, width, height, frameRate, true, decoder)) {
         SDL_DestroyWindow(window);
         SDL_QuitSubSystem(SDL_INIT_VIDEO);
         return false;
@@ -229,6 +240,8 @@ Session::Session(NvComputer* computer, NvApp& app)
       m_DecoderLock(0),
       m_NeedsIdr(false)
 {
+    qDebug() << "Server GPU:" << m_Computer->gpuModel;
+
     LiInitializeVideoCallbacks(&m_VideoCallbacks);
     m_VideoCallbacks.setup = drSetup;
     m_VideoCallbacks.submitDecodeUnit = drSubmitDecodeUnit;
@@ -237,7 +250,11 @@ Session::Session(NvComputer* computer, NvApp& app)
     m_VideoCallbacks.capabilities |= CAPABILITY_DIRECT_SUBMIT;
 
     // Slice up to 4 times for parallel decode, once slice per core
-    m_VideoCallbacks.capabilities |= CAPABILITY_SLICES_PER_FRAME(qMin(MAX_SLICES, SDL_GetCPUCount()));
+    int slices = qMin(MAX_SLICES, SDL_GetCPUCount());
+    m_VideoCallbacks.capabilities |= CAPABILITY_SLICES_PER_FRAME(slices);
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Encoder configured for %d slices per frame",
+                slices);
 
     LiInitializeStreamConfiguration(&m_StreamConfig);
     m_StreamConfig.width = m_Preferences.width;
@@ -245,6 +262,11 @@ Session::Session(NvComputer* computer, NvApp& app)
     m_StreamConfig.fps = m_Preferences.fps;
     m_StreamConfig.bitrate = m_Preferences.bitrateKbps;
     m_StreamConfig.hevcBitratePercentageMultiplier = 75;
+
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Video bitrate: %d kbps",
+                m_StreamConfig.bitrate);
+
     RAND_bytes(reinterpret_cast<unsigned char*>(m_StreamConfig.remoteInputAesKey),
                sizeof(m_StreamConfig.remoteInputAesKey));
 
@@ -254,6 +276,7 @@ Session::Session(NvComputer* computer, NvApp& app)
     switch (m_Preferences.audioConfig)
     {
     case StreamingPreferences::AC_AUTO:
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION, "Autodetecting audio configuration");
         m_StreamConfig.audioConfiguration = sdlDetermineAudioConfiguration();
         break;
     case StreamingPreferences::AC_FORCE_STEREO:
@@ -264,6 +287,10 @@ Session::Session(NvComputer* computer, NvApp& app)
         break;
     }
 
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Audio configuration: %d",
+                m_StreamConfig.audioConfiguration);
+
     switch (m_Preferences.videoCodecConfig)
     {
     case StreamingPreferences::VCC_AUTO:
@@ -305,10 +332,30 @@ Session::Session(NvComputer* computer, NvApp& app)
     }
 }
 
+void Session::emitLaunchWarning(QString text)
+{
+    // Emit the warning to the UI
+    emit displayLaunchWarning(text);
+
+    // Wait a little bit so the user can actually read what we just said.
+    // This wait is a little longer than the actual toast timeout (3 seconds)
+    // to allow it to transition off the screen before continuing.
+    uint32_t start = SDL_GetTicks();
+    while (!SDL_TICKS_PASSED(SDL_GetTicks(), start + 3500)) {
+        // Pump the UI loop while we wait
+        SDL_Delay(5);
+        QCoreApplication::processEvents(QEventLoop::ExcludeUserInputEvents);
+    }
+}
+
 bool Session::validateLaunch()
 {
     QStringList warningList;
 
+    if (m_Preferences.videoDecoderSelection == StreamingPreferences::VDS_FORCE_SOFTWARE) {
+        emitLaunchWarning("Your settings selection to force software decoding may cause poor streaming performance.");
+    }
+
     if (m_StreamConfig.supportsHevc) {
         bool hevcForced = m_Preferences.videoCodecConfig == StreamingPreferences::VCC_FORCE_HEVC ||
                 m_Preferences.videoCodecConfig == StreamingPreferences::VCC_FORCE_HEVC_HDR;
@@ -317,22 +364,26 @@ bool Session::validateLaunch()
                                        VIDEO_FORMAT_H265,
                                        m_StreamConfig.width,
                                        m_StreamConfig.height,
-                                       m_StreamConfig.fps)) {
+                                       m_StreamConfig.fps) &&
-            // NOTE: HEVC currently uses only 1 slice regardless of what
+                m_Preferences.videoDecoderSelection == StreamingPreferences::VDS_AUTO) {
-            // we provide in CAPABILITY_SLICES_PER_FRAME(), so we should
-            // never use it for software decoding (unless common-c starts
-            // respecting it for HEVC).
-            m_StreamConfig.supportsHevc = false;
-
             if (hevcForced) {
-                emit displayLaunchWarning("This PC's GPU doesn't support HEVC decoding.");
+                emitLaunchWarning("Using software decoding due to your selection to force HEVC without GPU support. This may cause poor streaming performance.");
+            }
+            else {
+                emitLaunchWarning("This PC's GPU doesn't support HEVC decoding.");
+                m_StreamConfig.supportsHevc = false;
             }
         }
 
         if (hevcForced) {
             if (m_Computer->maxLumaPixelsHEVC == 0) {
-                emit displayLaunchWarning("Your host PC GPU doesn't support HEVC. "
+                emitLaunchWarning("Your host PC GPU doesn't support HEVC. "
                                   "A GeForce GTX 900-series (Maxwell) or later GPU is required for HEVC streaming.");
+
+                // Moonlight-common-c will handle this case already, but we want
+                // to set this explicitly here so we can do our hardware acceleration
+                // check below.
+                m_StreamConfig.supportsHevc = false;
             }
         }
     }
@@ -343,11 +394,11 @@ bool Session::validateLaunch()
 
         // Check that the app supports HDR
         if (!m_App.hdrSupported) {
-            emit displayLaunchWarning(m_App.name + " doesn't support HDR10.");
+            emitLaunchWarning(m_App.name + " doesn't support HDR10.");
         }
         // Check that the server GPU supports HDR
         else if (!(m_Computer->serverCodecModeSupport & 0x200)) {
-            emit displayLaunchWarning("Your host PC GPU doesn't support HDR streaming. "
+            emitLaunchWarning("Your host PC GPU doesn't support HDR streaming. "
                               "A GeForce GTX 1000-series (Pascal) or later GPU is required for HDR streaming.");
         }
         else if (!isHardwareDecodeAvailable(m_Preferences.videoDecoderSelection,
@@ -355,7 +406,7 @@ bool Session::validateLaunch()
                                             m_StreamConfig.width,
                                             m_StreamConfig.height,
                                             m_StreamConfig.fps)) {
-            emit displayLaunchWarning("This PC's GPU doesn't support HEVC Main10 decoding for HDR streaming.");
+            emitLaunchWarning("This PC's GPU doesn't support HEVC Main10 decoding for HDR streaming.");
         }
         else {
             // TODO: Also validate display capabilites
@@ -368,7 +419,7 @@ bool Session::validateLaunch()
     if (m_StreamConfig.width >= 3840) {
         // Only allow 4K on GFE 3.x+
         if (m_Computer->gfeVersion.isNull() || m_Computer->gfeVersion.startsWith("2.")) {
-            emit displayLaunchWarning("GeForce Experience 3.0 or higher is required for 4K streaming.");
+            emitLaunchWarning("GeForce Experience 3.0 or higher is required for 4K streaming.");
 
             m_StreamConfig.width = 1920;
             m_StreamConfig.height = 1080;
@@ -376,7 +427,7 @@ bool Session::validateLaunch()
         // This list is sorted from least to greatest
         else if (m_Computer->displayModes.last().width < 3840 ||
                  (m_Computer->displayModes.last().refreshRate < 60 && m_StreamConfig.fps >= 60)) {
-            emit displayLaunchWarning("Your host PC GPU doesn't support 4K streaming. "
+            emitLaunchWarning("Your host PC GPU doesn't support 4K streaming. "
                               "A GeForce GTX 900-series (Maxwell) or later GPU is required for 4K streaming.");
 
             m_StreamConfig.width = 1920;
@@ -384,7 +435,23 @@ bool Session::validateLaunch()
         }
     }
 
-    // Always allow the launch to proceed for now
+    if (m_Preferences.videoDecoderSelection == StreamingPreferences::VDS_FORCE_HARDWARE &&
+            !isHardwareDecodeAvailable(m_Preferences.videoDecoderSelection,
+                                       m_StreamConfig.supportsHevc ? VIDEO_FORMAT_H265 : VIDEO_FORMAT_H264,
+                                       m_StreamConfig.width,
+                                       m_StreamConfig.height,
+                                       m_StreamConfig.fps)) {
+        if (m_Preferences.videoCodecConfig == StreamingPreferences::VCC_AUTO) {
+            emit displayLaunchError("Your selection to force hardware decoding cannot be satisfied due to missing hardware decoding support on this PC's GPU.");
+        }
+        else {
+            emit displayLaunchError("Your codec selection and force hardware decoding setting are not compatible. This PC's GPU lacks support for decoding your chosen codec.");
+        }
+
+        // Fail the launch, because we won't manage to get a decoder for the actual stream
+        return false;
+    }
+
     return true;
 }
 
@@ -410,38 +477,26 @@ void Session::getWindowDimensions(bool fullScreen,
 
     if (m_Window != nullptr) {
         displayIndex = SDL_GetWindowDisplayIndex(m_Window);
-        if (displayIndex < 0) {
+        SDL_assert(displayIndex >= 0);
-            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                         "SDL_GetWindowDisplayIndex() failed: %s",
-                         SDL_GetError());
-
-            // Assume display 0
-            displayIndex = 0;
     }
-    }
+    // If there's a display matching this exact resolution, pick that
-
+    // one (for native full-screen streaming). Otherwise, assume
-    if (fullScreen) {
+    // display 0 for now. TODO: Default to the screen that the Qt window is on
-        SDL_DisplayMode currentMode;
+    else if (fullScreen) {
-
+        for (int i = 0; i < SDL_GetNumVideoDisplays(); i++) {
-        if (SDL_GetCurrentDisplayMode(displayIndex, &currentMode) == 0) {
+            SDL_DisplayMode mode;
+            if (SDL_GetCurrentDisplayMode(i, &mode) == 0 &&
+                    m_ActiveVideoWidth == mode.w &&
+                    m_ActiveVideoHeight == mode.h) {
                 SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
-                        "Using current display mode: %dx%dx%d",
+                            "Found exact resolution match on display: %d",
-                        currentMode.w, currentMode.h, currentMode.refresh_rate);
+                            i);
-            width = currentMode.w;
+                displayIndex = i;
-            height = currentMode.h;
+                break;
+            }
         }
-        else {
-            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                         "Unable to get current display mode: %s",
-                         SDL_GetError());
-            width = m_StreamConfig.width;
-            height = m_StreamConfig.height;
     }
 
-        // Full-screen modes always start at the origin
-        x = y = SDL_WINDOWPOS_UNDEFINED_DISPLAY(displayIndex);
-    }
-    else {
     SDL_Rect usableBounds;
     if (SDL_GetDisplayUsableBounds(displayIndex, &usableBounds) == 0) {
         width = usableBounds.w;
@@ -463,6 +518,13 @@ void Session::getWindowDimensions(bool fullScreen,
 
             width -= left + right;
             height -= top + bottom;
+
+            // If the stream window can fit within the usable drawing area with 1:1
+            // scaling, do that rather than filling the screen.
+            if (m_StreamConfig.width < width && m_StreamConfig.height < height) {
+                width = m_StreamConfig.width;
+                height = m_StreamConfig.height;
+            }
         }
     }
     else {
@@ -475,7 +537,6 @@ void Session::getWindowDimensions(bool fullScreen,
         x = y = SDL_WINDOWPOS_UNDEFINED_DISPLAY(displayIndex);
     }
 }
-}
 
 void Session::toggleFullscreen()
 {
@@ -487,6 +548,7 @@ void Session::toggleFullscreen()
     // that way we have accurate display size metrics (not the size our stream changed it to).
     if (!fullScreen) {
         SDL_SetWindowFullscreen(m_Window, 0);
+        SDL_SetWindowResizable(m_Window, SDL_TRUE);
     }
 
     getWindowDimensions(fullScreen, x, y, width, height);
@@ -495,6 +557,7 @@ void Session::toggleFullscreen()
     SDL_SetWindowSize(m_Window, width, height);
 
     if (fullScreen) {
+        SDL_SetWindowResizable(m_Window, SDL_FALSE);
         SDL_SetWindowFullscreen(m_Window, SDL_OS_FULLSCREEN_FLAG);
     }
 }
@@ -525,6 +588,18 @@ void Session::exec()
     Q_ASSERT(m_Computer->currentGameId == 0 ||
              m_Computer->currentGameId == m_App.id);
 
+    bool enableGameOptimizations = false;
+    for (const NvDisplayMode &mode : m_Computer->displayModes) {
+        if (mode.width == m_StreamConfig.width &&
+                mode.height == m_StreamConfig.height) {
+            SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                        "Found host supported resolution: %dx%d",
+                        mode.width, mode.height);
+            enableGameOptimizations = prefs.gameOptimizations;
+            break;
+        }
+    }
+
     try {
         NvHTTP http(m_Computer->activeAddress);
         if (m_Computer->currentGameId != 0) {
@@ -532,7 +607,7 @@ void Session::exec()
         }
         else {
             http.launchApp(m_App.id, &m_StreamConfig,
-                           prefs.gameOptimizations,
+                           enableGameOptimizations,
                            prefs.playAudioOnHost,
                            inputHandler.getAttachedGamepadMask());
         }
@@ -574,8 +649,8 @@ void Session::exec()
     if (err != 0) {
         // We already displayed an error dialog in the stage failure
         // listener.
-        s_ActiveSessionSemaphore.release();
         SDL_QuitSubSystem(SDL_INIT_VIDEO);
+        s_ActiveSessionSemaphore.release();
         return;
     }
 
@@ -592,14 +667,13 @@ void Session::exec()
                                 y,
                                 width,
                                 height,
-                                m_Preferences.fullScreen ?
+                                0);
-                                    SDL_OS_FULLSCREEN_FLAG : 0);
     if (!m_Window) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                      "SDL_CreateWindow() failed: %s",
                      SDL_GetError());
-        s_ActiveSessionSemaphore.release();
         SDL_QuitSubSystem(SDL_INIT_VIDEO);
+        s_ActiveSessionSemaphore.release();
         return;
     }
 
@@ -611,6 +685,20 @@ void Session::exec()
 
         SDL_SetWindowPosition(m_Window, x, y);
         SDL_SetWindowSize(m_Window, width, height);
+
+        // Passing SDL_WINDOW_RESIZABLE to set this during window
+        // creation causes our window to be full screen for some reason
+        SDL_SetWindowResizable(m_Window, SDL_TRUE);
+    }
+    else {
+        // Update the window display mode based on our current monitor
+        SDL_DisplayMode mode;
+        if (SDL_GetDesktopDisplayMode(SDL_GetWindowDisplayIndex(m_Window), &mode) == 0) {
+            SDL_SetWindowDisplayMode(m_Window, &mode);
+        }
+
+        // Enter full screen
+        SDL_SetWindowFullscreen(m_Window, SDL_OS_FULLSCREEN_FLAG);
     }
 
     QSvgRenderer svgIconRenderer(QString(":/res/moonlight.svg"));
@@ -636,12 +724,31 @@ void Session::exec()
     // Capture the mouse by default on release builds only.
     // This prevents the mouse from becoming trapped inside
     // Moonlight when it's halted at a debug break.
+    if (m_Preferences.fullScreen) {
         SDL_SetRelativeMouseMode(SDL_TRUE);
+    }
 #endif
 
+    // Stop text input. SDL enables it by default
+    // when we initialize the video subsystem, but this
+    // causes an IME popup when certain keys are held down
+    // on macOS.
+    SDL_StopTextInput();
+
     // Disable the screen saver
     SDL_DisableScreenSaver();
 
+#ifdef Q_OS_WIN32
+    // HACK: SDL doesn't call this, so we must do so to disable the
+    // screensaver when we're in windowed mode. DirectX will disable
+    // it for us when we're in full-screen exclusive mode.
+    SetThreadExecutionState(ES_CONTINUOUS | ES_DISPLAY_REQUIRED);
+#endif
+
+    // Set timer resolution to 1 ms on Windows for greater
+    // sleep precision and more accurate callback timing.
+    SDL_SetHint(SDL_HINT_TIMER_RESOLUTION, "1");
+
     // Raise the priority of the main thread, since it handles
     // time-sensitive video rendering
     if (SDL_SetThreadPriority(SDL_THREAD_PRIORITY_HIGH) < 0) {
@@ -653,7 +760,15 @@ void Session::exec()
     // Hijack this thread to be the SDL main thread. We have to do this
     // because we want to suspend all Qt processing until the stream is over.
     SDL_Event event;
-    while (SDL_WaitEvent(&event)) {
+    for (;;) {
+        // We explicitly use SDL_PollEvent() and SDL_Delay() because
+        // SDL_WaitEvent() has an internal SDL_Delay(10) inside which
+        // blocks this thread too long for high polling rate mice and high
+        // refresh rate displays.
+        if (!SDL_PollEvent(&event)) {
+            SDL_Delay(1);
+            continue;
+        }
         switch (event.type) {
         case SDL_QUIT:
             SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
@@ -690,20 +805,41 @@ void Session::exec()
             // Fall through
         case SDL_RENDER_DEVICE_RESET:
         case SDL_RENDER_TARGETS_RESET:
+
             SDL_AtomicLock(&m_DecoderLock);
 
             // Destroy the old decoder
             delete m_VideoDecoder;
 
-            // Chose a new decoder (hopefully the same one, but possibly
+            // Flush any other pending window events that could
+            // send us back here immediately
+            SDL_PumpEvents();
+            SDL_FlushEvent(SDL_WINDOWEVENT);
+
+            // Update the window display mode based on our current monitor
+            SDL_DisplayMode mode;
+            if (SDL_GetDesktopDisplayMode(SDL_GetWindowDisplayIndex(m_Window), &mode) == 0) {
+                SDL_SetWindowDisplayMode(m_Window, &mode);
+            }
+
+            // Now that the old decoder is dead, flush any events it may
+            // have queued to reset itself (if this reset was the result
+            // of state loss).
+            SDL_PumpEvents();
+            SDL_FlushEvent(SDL_RENDER_DEVICE_RESET);
+            SDL_FlushEvent(SDL_RENDER_TARGETS_RESET);
+
+            // Choose a new decoder (hopefully the same one, but possibly
             // not if a GPU was removed or something).
             if (!chooseDecoder(m_Preferences.videoDecoderSelection,
                                m_Window, m_ActiveVideoFormat, m_ActiveVideoWidth,
                                m_ActiveVideoHeight, m_ActiveVideoFrameRate,
+                               m_Preferences.enableVsync,
                                s_ActiveSession->m_VideoDecoder)) {
                 SDL_AtomicUnlock(&m_DecoderLock);
                 SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                              "Failed to recreate decoder after reset");
+                emit displayLaunchError("Unable to initialize video decoder. Please check your streaming settings and try again.");
                 goto DispatchDeferredCleanup;
             }
 
@@ -746,10 +882,16 @@ void Session::exec()
                  SDL_GetError());
 
 DispatchDeferredCleanup:
+#ifdef Q_OS_WIN32
+    // HACK: See comment above
+    SetThreadExecutionState(ES_CONTINUOUS);
+#endif
+
     // Uncapture the mouse and hide the window immediately,
     // so we can return to the Qt GUI ASAP.
     SDL_SetRelativeMouseMode(SDL_FALSE);
     SDL_EnableScreenSaver();
+    SDL_SetHint(SDL_HINT_TIMER_RESOLUTION, "0");
 
     // Destroy the decoder, since this must be done on the main thread
     SDL_AtomicLock(&m_DecoderLock);

app/streaming/session.hpp

@@ -39,6 +39,8 @@ signals:
 private:
     bool validateLaunch();
 
+    void emitLaunchWarning(QString text);
+
     int getDecoderCapabilities();
 
     int sdlDetermineAudioConfiguration();
@@ -52,7 +54,7 @@ private:
     static
     bool chooseDecoder(StreamingPreferences::VideoDecoderSelection vds,
                        SDL_Window* window, int videoFormat, int width, int height,
-                       int frameRate, IVideoDecoder*& chosenDecoder);
+                       int frameRate, bool enableVsync, IVideoDecoder*& chosenDecoder);
 
     static
     void clStageStarting(int stage);

app/streaming/streamutils.cpp

@@ -0,0 +1,20 @@
+#include "streamutils.h"
+
+void StreamUtils::scaleSourceToDestinationSurface(SDL_Rect* src, SDL_Rect* dst)
+{
+    int dstH = dst->w * src->h / src->w;
+    int dstW = dst->h * src->w / src->h;
+
+    if (dstH > dst->h) {
+        dst->y = 0;
+        dst->x = (dst->w - dstW) / 2;
+        dst->w = dstW;
+        SDL_assert(dst->w * src->h / src->w <= dst->h);
+    }
+    else {
+        dst->x = 0;
+        dst->y = (dst->h - dstH) / 2;
+        dst->h = dstH;
+        SDL_assert(dst->h * src->w / src->h <= dst->w);
+    }
+}

app/streaming/streamutils.h

@@ -0,0 +1,10 @@
+#pragma once
+
+#include <SDL.h>
+
+class StreamUtils
+{
+public:
+    static
+    void scaleSourceToDestinationSurface(SDL_Rect* src, SDL_Rect* dst);
+};

app/streaming/video/decoder.h

@@ -16,7 +16,8 @@ public:
                             int videoFormat,
                             int width,
                             int height,
-                            int frameRate) = 0;
+                            int frameRate,
+                            bool enableVsync) = 0;
     virtual bool isHardwareAccelerated() = 0;
     virtual int submitDecodeUnit(PDECODE_UNIT du) = 0;
     virtual void renderFrame(SDL_UserEvent* event) = 0;

app/streaming/video/ffmpeg-renderers/dxva2.cpp

@@ -1,5 +1,14 @@
 #include <initguid.h>
 #include "dxva2.h"
+#include "../ffmpeg.h"
+#include <streaming/streamutils.h>
+
+#include <SDL_syswm.h>
+
+#define WIN32_LEAN_AND_MEAN
+#include <Windows.h>
+#include <VersionHelpers.h>
+#include <dwmapi.h>
 
 #include <Limelight.h>
 
@@ -8,7 +17,6 @@ DEFINE_GUID(DXVADDI_Intel_ModeH264_E, 0x604F8E68,0x4951,0x4C54,0x88,0xFE,0xAB,0x
 #define SAFE_COM_RELEASE(x) if (x) { (x)->Release(); }
 
 DXVA2Renderer::DXVA2Renderer() :
-    m_SdlRenderer(nullptr),
     m_DecService(nullptr),
     m_Decoder(nullptr),
     m_SurfacesUsed(0),
@@ -21,10 +29,15 @@ DXVA2Renderer::DXVA2Renderer() :
 {
     RtlZeroMemory(m_DecSurfaces, sizeof(m_DecSurfaces));
     RtlZeroMemory(&m_DXVAContext, sizeof(m_DXVAContext));
+
+    // Use MMCSS scheduling for lower scheduling latency while we're streaming
+    DwmEnableMMCSS(TRUE);
 }
 
 DXVA2Renderer::~DXVA2Renderer()
 {
+    DwmEnableMMCSS(FALSE);
+
     SAFE_COM_RELEASE(m_DecService);
     SAFE_COM_RELEASE(m_Decoder);
     SAFE_COM_RELEASE(m_Device);
@@ -39,10 +52,6 @@ DXVA2Renderer::~DXVA2Renderer()
     if (m_Pool != nullptr) {
         av_buffer_pool_uninit(&m_Pool);
     }
-
-    if (m_SdlRenderer != nullptr) {
-        SDL_DestroyRenderer(m_SdlRenderer);
-    }
 }
 
 void DXVA2Renderer::ffPoolDummyDelete(void*, uint8_t*)
@@ -75,11 +84,8 @@ bool DXVA2Renderer::prepareDecoderContext(AVCodecContext* context)
 
     context->hwaccel_context = &m_DXVAContext;
 
-    context->get_format = ffGetFormat;
     context->get_buffer2 = ffGetBuffer2;
 
-    context->opaque = this;
-
     m_Pool = av_buffer_pool_init2(ARRAYSIZE(m_DecSurfaces), this, ffPoolAlloc, nullptr);
     if (!m_Pool) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
@@ -87,12 +93,15 @@ bool DXVA2Renderer::prepareDecoderContext(AVCodecContext* context)
         return false;
     }
 
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Using DXVA2 accelerated renderer");
+
     return true;
 }
 
 int DXVA2Renderer::ffGetBuffer2(AVCodecContext* context, AVFrame* frame, int)
 {
-    DXVA2Renderer* me = reinterpret_cast<DXVA2Renderer*>(context->opaque);
+    DXVA2Renderer* me = (DXVA2Renderer*)((FFmpegVideoDecoder*)context->opaque)->getRenderer();
 
     frame->buf[0] = av_buffer_pool_get(me->m_Pool);
     if (!frame->buf[0]) {
@@ -107,25 +116,14 @@ int DXVA2Renderer::ffGetBuffer2(AVCodecContext* context, AVFrame* frame, int)
     return 0;
 }
 
-enum AVPixelFormat DXVA2Renderer::ffGetFormat(AVCodecContext*,
-                                              const enum AVPixelFormat* pixFmts)
-{
-    const enum AVPixelFormat *p;
-
-    for (p = pixFmts; *p != -1; p++) {
-        if (*p == AV_PIX_FMT_DXVA2_VLD)
-            return *p;
-    }
-
-    SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                 "Failed to find DXVA2 HW surface format");
-    return AV_PIX_FMT_NONE;
-}
-
 bool DXVA2Renderer::initializeDecoder()
 {
     HRESULT hr;
 
+    if (isDecoderBlacklisted()) {
+        return false;
+    }
+
     hr = DXVA2CreateVideoService(m_Device, IID_IDirectXVideoDecoderService,
                                  reinterpret_cast<void**>(&m_DecService));
     if (FAILED(hr)) {
@@ -208,18 +206,11 @@ bool DXVA2Renderer::initializeDecoder()
         return false;
     }
 
-    int alignment;
+    // Alignment was already taken care of
-
+    SDL_assert(m_Desc.SampleWidth % 16 == 0);
-    // HEVC using DXVA requires 128B alignment
+    SDL_assert(m_Desc.SampleHeight % 16 == 0);
-    if (m_VideoFormat & VIDEO_FORMAT_MASK_H265) {
+    hr = m_DecService->CreateSurface(m_Desc.SampleWidth,
-        alignment = 128;
+                                     m_Desc.SampleHeight,
-    }
-    else {
-        alignment = 16;
-    }
-
-    hr = m_DecService->CreateSurface(FFALIGN(m_VideoWidth, alignment),
-                                     FFALIGN(m_VideoHeight, alignment),
                                      ARRAYSIZE(m_DecSurfaces) - 1,
                                      m_Desc.Format,
                                      D3DPOOL_DEFAULT,
@@ -349,37 +340,247 @@ bool DXVA2Renderer::initializeRenderer()
     return true;
 }
 
-bool DXVA2Renderer::initialize(SDL_Window* window, int videoFormat, int width, int height)
+bool DXVA2Renderer::isDecoderBlacklisted()
+{
+    IDirect3D9* d3d9;
+    HRESULT hr;
+    bool result = false;
+
+    // TODO: Update for HEVC Main10
+    SDL_assert(m_VideoFormat != VIDEO_FORMAT_H265_MAIN10);
+
+    hr = m_Device->GetDirect3D(&d3d9);
+    if (SUCCEEDED(hr)) {
+        D3DCAPS9 caps;
+
+        hr = m_Device->GetDeviceCaps(&caps);
+        if (SUCCEEDED(hr)) {
+            D3DADAPTER_IDENTIFIER9 id;
+
+            hr = d3d9->GetAdapterIdentifier(caps.AdapterOrdinal, 0, &id);
+            if (SUCCEEDED(hr)) {
+                SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                            "Detected GPU: %s (%x:%x)",
+                            id.Description,
+                            id.VendorId,
+                            id.DeviceId);
+
+                if (id.VendorId == 0x8086) {
+                    // Intel seems to encode the series in the high byte of
+                    // the device ID. We want to avoid the "Partial" acceleration
+                    // support explicitly. Those will claim to have HW acceleration
+                    // but perform badly.
+                    // https://en.wikipedia.org/wiki/Intel_Graphics_Technology#Capabilities_(GPU_video_acceleration)
+                    // https://raw.githubusercontent.com/GameTechDev/gpudetect/master/IntelGfx.cfg
+                    switch (id.DeviceId & 0xFF00) {
+                    case 0x0400: // Haswell
+                    case 0x0A00: // Haswell
+                    case 0x0D00: // Haswell
+                    case 0x1600: // Broadwell
+                    case 0x2200: // Cherry Trail and Braswell
+                        // Blacklist these for HEVC to avoid hybrid decode
+                        result = (m_VideoFormat & VIDEO_FORMAT_MASK_H265) != 0;
+                        break;
+                    default:
+                        // Everything else is fine with whatever it says it supports
+                        result = false;
+                        break;
+                    }
+                }
+                else if (id.VendorId == 0x10DE) {
+                    // For NVIDIA, we wait to avoid those GPUs with Feature Set E
+                    // for HEVC decoding, since that's hybrid.
+                    // https://en.wikipedia.org/wiki/Nvidia_PureVideo
+                    // http://envytools.readthedocs.io/en/latest/hw/pciid.html (missing GM200)
+                    if ((id.DeviceId >= 0x1340 && id.DeviceId <= 0x137F) || // GM108
+                            (id.DeviceId >= 0x1380 && id.DeviceId <= 0x13BF) || // GM107
+                            (id.DeviceId >= 0x13C0 && id.DeviceId <= 0x13FF) || // GM204
+                            (id.DeviceId >= 0x1617 && id.DeviceId <= 0x161A) || // GM204
+                            (id.DeviceId == 0x1667) || // GM204
+                            (id.DeviceId >= 0x17C0 && id.DeviceId <= 0x17FF)) { // GM200
+                        // Avoid HEVC on Feature Set E GPUs
+                        result = (m_VideoFormat & VIDEO_FORMAT_MASK_H265) != 0;
+                    }
+                    else {
+                        result = false;
+                    }
+                }
+                else if (id.VendorId == 0x1002) {
+                    // AMD doesn't seem to do hybrid acceleration?
+                    result = false;
+                }
+                else {
+                    SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                                "Unrecognized vendor ID: %x",
+                                id.VendorId);
+                }
+            }
+        }
+        else {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "GetDeviceCaps() failed: %x", hr);
+        }
+
+        d3d9->Release();
+    }
+    else {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "GetDirect3D() failed: %x", hr);
+    }
+
+    if (result) {
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "GPU blacklisted for format %x",
+                    m_VideoFormat);
+    }
+
+    return result;
+}
+
+bool DXVA2Renderer::initializeDevice(SDL_Window* window, bool enableVsync)
+{
+    SDL_SysWMinfo info;
+
+    SDL_VERSION(&info.version);
+    SDL_GetWindowWMInfo(window, &info);
+
+    IDirect3D9Ex* d3d9ex;
+    HRESULT hr = Direct3DCreate9Ex(D3D_SDK_VERSION, &d3d9ex);
+    if (FAILED(hr)) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Direct3DCreate9Ex() failed: %x",
+                     hr);
+        return false;
+    }
+
+    int adapterIndex = SDL_Direct3D9GetAdapterIndex(SDL_GetWindowDisplayIndex(window));
+    Uint32 windowFlags = SDL_GetWindowFlags(window);
+
+    D3DCAPS9 deviceCaps;
+    d3d9ex->GetDeviceCaps(adapterIndex, D3DDEVTYPE_HAL, &deviceCaps);
+
+    D3DDISPLAYMODEEX currentMode;
+    currentMode.Size = sizeof(currentMode);
+    d3d9ex->GetAdapterDisplayModeEx(adapterIndex, &currentMode, nullptr);
+
+    D3DPRESENT_PARAMETERS d3dpp = {};
+    d3dpp.hDeviceWindow = info.info.win.window;
+    d3dpp.Flags = D3DPRESENTFLAG_VIDEO;
+    d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;
+
+    if ((windowFlags & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN) {
+        d3dpp.Windowed = false;
+        d3dpp.BackBufferWidth = currentMode.Width;
+        d3dpp.BackBufferHeight = currentMode.Height;
+        d3dpp.FullScreen_RefreshRateInHz = currentMode.RefreshRate;
+        d3dpp.BackBufferFormat = currentMode.Format;
+    }
+    else {
+        d3dpp.Windowed = true;
+        d3dpp.BackBufferFormat = D3DFMT_UNKNOWN;
+
+        SDL_GetWindowSize(window, (int*)&d3dpp.BackBufferWidth, (int*)&d3dpp.BackBufferHeight);
+    }
+
+    BOOL dwmEnabled;
+    DwmIsCompositionEnabled(&dwmEnabled);
+    if (d3dpp.Windowed && dwmEnabled) {
+        // If composition enabled, disable v-sync and let DWM manage things
+        // to reduce latency by avoiding double v-syncing.
+        d3dpp.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE;
+
+        // D3DSWAPEFFECT_FLIPEX requires at least 2 back buffers to allow us to
+        // continue while DWM is waiting to render the surface to the display.
+        d3dpp.SwapEffect = D3DSWAPEFFECT_FLIPEX;
+        d3dpp.BackBufferCount = 2;
+
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "Windowed mode with DWM running");
+    }
+    else if (enableVsync) {
+        // Uncomposited desktop or full-screen exclusive mode with V-sync enabled
+        // We will enable V-sync in this scenario to avoid tearing.
+        d3dpp.PresentationInterval = D3DPRESENT_INTERVAL_ONE;
+        d3dpp.BackBufferCount = 1;
+
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "V-Sync enabled");
+    }
+    else {
+        // Uncomposited desktop or full-screen exclusive mode with V-sync disabled
+        // We will allowing tearing for lowest latency.
+        d3dpp.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE;
+        d3dpp.BackBufferCount = 1;
+
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "V-Sync disabled in tearing mode");
+    }
+
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Windowed: %d | Present Interval: %x",
+                d3dpp.Windowed, d3dpp.PresentationInterval);
+
+    // FFmpeg requires this attribute for doing asynchronous decoding
+    // in a separate thread with this device.
+    int deviceFlags = D3DCREATE_MULTITHREADED;
+
+    if (deviceCaps.DevCaps & D3DDEVCAPS_HWTRANSFORMANDLIGHT) {
+        deviceFlags |= D3DCREATE_HARDWARE_VERTEXPROCESSING;
+    }
+    else {
+        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                    "No hardware vertex processing support!");
+        deviceFlags |= D3DCREATE_SOFTWARE_VERTEXPROCESSING;
+    }
+
+    hr = d3d9ex->CreateDeviceEx(adapterIndex,
+                                D3DDEVTYPE_HAL,
+                                d3dpp.hDeviceWindow,
+                                deviceFlags,
+                                &d3dpp,
+                                d3dpp.Windowed ? nullptr : &currentMode,
+                                &m_Device);
+
+    d3d9ex->Release();
+
+    if (FAILED(hr)) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "CreateDeviceEx() failed: %x",
+                     hr);
+        return false;
+    }
+
+    hr = m_Device->SetMaximumFrameLatency(1);
+    if (FAILED(hr)) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "SetMaximumFrameLatency() failed: %x",
+                     hr);
+        return false;
+    }
+
+    return true;
+}
+
+bool DXVA2Renderer::initialize(SDL_Window* window, int videoFormat, int width, int height, int, bool enableVsync)
 {
     m_VideoFormat = videoFormat;
     m_VideoWidth = width;
     m_VideoHeight = height;
 
-    // FFmpeg will be decoding on different threads than the main thread that we're
+    RtlZeroMemory(&m_Desc, sizeof(m_Desc));
-    // currently running on right now. We must set this hint so SDL will pass
-    // D3DCREATE_MULTITHREADED to IDirect3D9::CreateDevice().
-    SDL_SetHint(SDL_HINT_RENDER_DIRECT3D_THREADSAFE, "1");
 
-    // We require full-screen desktop mode to avoid having to enable V-sync
+    int alignment;
-    // to synchronize frame delivery (which has a much higher latency penalty).
+
-    m_SdlRenderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
+    // HEVC using DXVA requires 128B alignment
-    if (!m_SdlRenderer) {
+    if (m_VideoFormat & VIDEO_FORMAT_MASK_H265) {
-        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+        alignment = 128;
-                     "SDL_CreateRenderer() failed: %s",
+    }
-                     SDL_GetError());
+    else {
-        return false;
+        alignment = 16;
     }
 
-    m_Device = SDL_RenderGetD3D9Device(m_SdlRenderer);
+    m_Desc.SampleWidth = FFALIGN(m_VideoWidth, alignment);
-
+    m_Desc.SampleHeight = FFALIGN(m_VideoHeight, alignment);
-    // Draw a black frame until the video stream starts rendering
-    SDL_SetRenderDrawColor(m_SdlRenderer, 0, 0, 0, SDL_ALPHA_OPAQUE);
-    SDL_RenderClear(m_SdlRenderer);
-    SDL_RenderPresent(m_SdlRenderer);
-
-    RtlZeroMemory(&m_Desc, sizeof(m_Desc));
-    m_Desc.SampleWidth = m_VideoWidth;
-    m_Desc.SampleHeight = m_VideoHeight;
     m_Desc.SampleFormat.VideoChromaSubsampling = DXVA2_VideoChromaSubsampling_Unknown;
     m_Desc.SampleFormat.NominalRange = DXVA2_NominalRange_Unknown;
     m_Desc.SampleFormat.VideoTransferMatrix = DXVA2_VideoTransferMatrix_Unknown;
@@ -389,6 +590,10 @@ bool DXVA2Renderer::initialize(SDL_Window* window, int videoFormat, int width, i
     m_Desc.SampleFormat.SampleFormat = DXVA2_SampleProgressiveFrame;
     m_Desc.Format = (D3DFORMAT)MAKEFOURCC('N','V','1','2');
 
+    if (!initializeDevice(window, enableVsync)) {
+        return false;
+    }
+
     if (!initializeDecoder()) {
         return false;
     }
@@ -397,10 +602,24 @@ bool DXVA2Renderer::initialize(SDL_Window* window, int videoFormat, int width, i
         return false;
     }
 
+    // For some reason, using Direct3D9Ex breaks this with multi-monitor setups.
+    // When focus is lost, the window is minimized then immediately restored without
+    // input focus. This glitches out the renderer and a bunch of other stuff.
+    // Direct3D9Ex itself seems to have this minimize on focus loss behavior on its
+    // own, so just disable SDL's handling of the focus loss event.
+    SDL_SetHint(SDL_HINT_VIDEO_MINIMIZE_ON_FOCUS_LOSS, "0");
+
     return true;
 }
 
-void DXVA2Renderer::renderFrame(AVFrame* frame)
+bool DXVA2Renderer::needsTestFrame()
+{
+    // We validate the DXVA2 profiles are supported
+    // in initialize() so no test frame is required
+    return false;
+}
+
+void DXVA2Renderer::renderFrameAtVsync(AVFrame *frame)
 {
     IDirect3DSurface9* surface = reinterpret_cast<IDirect3DSurface9*>(frame->data[3]);
     HRESULT hr;
@@ -500,28 +719,26 @@ void DXVA2Renderer::renderFrame(AVFrame* frame)
     sample.Start = m_FrameIndex;
     sample.End = m_FrameIndex + 1;
     sample.SrcSurface = surface;
-    sample.SrcRect.right = m_Desc.SampleWidth;
+    sample.SrcRect.right = m_VideoWidth;
-    sample.SrcRect.bottom = m_Desc.SampleHeight;
+    sample.SrcRect.bottom = m_VideoHeight;
     sample.SampleFormat = m_Desc.SampleFormat;
     sample.PlanarAlpha = DXVA2_Fixed32OpaqueAlpha();
 
     // Center in frame and preserve aspect ratio
-    double srcAspectRatio = (double)m_Desc.SampleWidth / (double)m_Desc.SampleHeight;
+    SDL_Rect src, dst;
-    double dstAspectRatio = (double)m_DisplayWidth / (double)m_DisplayHeight;
+    src.x = src.y = 0;
-    if (dstAspectRatio < srcAspectRatio) {
+    src.w = m_VideoWidth;
-        // Greater height per width
+    src.h = m_VideoHeight;
-        int drawHeight = (int)(m_DisplayWidth / srcAspectRatio);
+    dst.x = dst.y = 0;
-        sample.DstRect.top = (m_DisplayHeight - drawHeight) / 2;
+    dst.w = m_DisplayWidth;
-        sample.DstRect.bottom = sample.DstRect.bottom + drawHeight;
+    dst.h = m_DisplayHeight;
-        sample.DstRect.right = m_DisplayWidth;
+
-    }
+    StreamUtils::scaleSourceToDestinationSurface(&src, &dst);
-    else {
+
-        // Greater width per height
+    sample.DstRect.left = dst.x;
-        int drawWidth = (int)(m_DisplayHeight * srcAspectRatio);
+    sample.DstRect.right = dst.x + dst.w;
-        sample.DstRect.bottom = m_DisplayHeight;
+    sample.DstRect.top = dst.y;
-        sample.DstRect.left = (m_DisplayWidth - drawWidth) / 2;
+    sample.DstRect.bottom = dst.y + dst.h;
-        sample.DstRect.right = sample.DstRect.left + drawWidth;
-    }
 
     DXVA2_VideoProcessBltParams bltParams = {};
 
@@ -538,33 +755,27 @@ void DXVA2Renderer::renderFrame(AVFrame* frame)
 
     bltParams.Alpha = DXVA2_Fixed32OpaqueAlpha();
 
-    if (SDL_RenderClear(m_SdlRenderer) != 0) {
+    m_Device->Clear(0, nullptr, D3DCLEAR_TARGET, D3DCOLOR_ARGB(255, 0, 0, 0), 0.0f, 0);
-        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
-                    "SDL_RenderClear() failed: %s",
-                    SDL_GetError());
-        av_frame_free(&frame);
-
-        // We're going to cheat a little bit here. It seems SDL's
-        // renderer may flake out in scenarios like moving the window
-        // between monitors, so generate a synthetic reset event for
-        // the main loop to consume.
-        SDL_Event event;
-        event.type = SDL_RENDER_TARGETS_RESET;
-        SDL_PushEvent(&event);
-
-        return;
-    }
 
     hr = m_Processor->VideoProcessBlt(m_RenderTarget, &bltParams, &sample, 1, nullptr);
-    av_frame_free(&frame);
-
     if (FAILED(hr)) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                      "VideoProcessBlt() failed: %x",
                      hr);
+        SDL_Event event;
+        event.type = SDL_RENDER_TARGETS_RESET;
+        SDL_PushEvent(&event);
+        return;
     }
 
-    // We must try to present to trigger SDL's logic to recover the render target,
+    hr = m_Device->PresentEx(nullptr, nullptr, nullptr, nullptr, 0);
-    // even if VideoProcessBlt() fails.
+    if (FAILED(hr)) {
-    SDL_RenderPresent(m_SdlRenderer);
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "PresentEx() failed: %x",
+                     hr);
+        SDL_Event event;
+        event.type = SDL_RENDER_TARGETS_RESET;
+        SDL_PushEvent(&event);
+        return;
+    }
 }

app/streaming/video/ffmpeg-renderers/dxva2.h

@@ -1,6 +1,7 @@
 #pragma once
 
 #include "renderer.h"
+#include "pacer/pacer.h"
 
 #include <d3d9.h>
 #include <dxva2api.h>
@@ -17,13 +18,18 @@ public:
     virtual bool initialize(SDL_Window* window,
                             int videoFormat,
                             int width,
-                            int height);
+                            int height,
+                            int maxFps,
+                            bool enableVsync);
     virtual bool prepareDecoderContext(AVCodecContext* context);
-    virtual void renderFrame(AVFrame* frame);
+    virtual void renderFrameAtVsync(AVFrame* frame);
+    virtual bool needsTestFrame();
 
 private:
     bool initializeDecoder();
     bool initializeRenderer();
+    bool initializeDevice(SDL_Window* window, bool enableVsync);
+    bool isDecoderBlacklisted();
 
     static
     AVBufferRef* ffPoolAlloc(void* opaque, int size);
@@ -34,10 +40,6 @@ private:
     static
     int ffGetBuffer2(AVCodecContext* context, AVFrame* frame, int flags);
 
-    static
-    enum AVPixelFormat ffGetFormat(AVCodecContext*,
-                                   const enum AVPixelFormat* pixFmts);
-
     int m_VideoFormat;
     int m_VideoWidth;
     int m_VideoHeight;
@@ -45,8 +47,6 @@ private:
     int m_DisplayWidth;
     int m_DisplayHeight;
 
-    SDL_Renderer* m_SdlRenderer;
-
     struct dxva_context m_DXVAContext;
     IDirect3DSurface9* m_DecSurfaces[19];
     DXVA2_ConfigPictureDecode m_Config;
@@ -55,7 +55,7 @@ private:
     int m_SurfacesUsed;
     AVBufferPool* m_Pool;
 
-    IDirect3DDevice9* m_Device;
+    IDirect3DDevice9Ex* m_Device;
     IDirect3DSurface9* m_RenderTarget;
     IDirectXVideoProcessorService* m_ProcService;
     IDirectXVideoProcessor* m_Processor;

app/streaming/video/ffmpeg-renderers/pacer/displaylinkvsyncsource.cpp

@@ -0,0 +1,50 @@
+#include "displaylinkvsyncsource.h"
+
+DisplayLinkVsyncSource::DisplayLinkVsyncSource(Pacer* pacer)
+    : m_Pacer(pacer),
+      m_DisplayLink(nullptr)
+{
+
+}
+
+DisplayLinkVsyncSource::~DisplayLinkVsyncSource()
+{
+    if (m_DisplayLink != nullptr) {
+        CVDisplayLinkStop(m_DisplayLink);
+        CVDisplayLinkRelease(m_DisplayLink);
+    }
+}
+
+CVReturn
+DisplayLinkVsyncSource::displayLinkOutputCallback(
+    CVDisplayLinkRef,
+    const CVTimeStamp* /* now */,
+    const CVTimeStamp* /* vsyncTime */,
+    CVOptionFlags,
+    CVOptionFlags*,
+    void *displayLinkContext)
+{
+    auto me = reinterpret_cast<DisplayLinkVsyncSource*>(displayLinkContext);
+
+    // In my testing on macOS 10.13, this callback is invoked about 24 ms
+    // prior to the specified v-sync time (now - vsyncTime). Since this is
+    // greater than the standard v-sync interval (16 ms = 60 FPS), we will
+    // draw using the current host time, rather than the actual v-sync target
+    // time. Because the CVDisplayLink is in sync with the actual v-sync
+    // interval, even if many ms prior, we can safely use the current host time
+    // and get a consistent callback for each v-sync. This reduces video latency
+    // by at least 1 frame vs. rendering with the actual vsyncTime.
+    me->m_Pacer->vsyncCallback(500 / me->m_DisplayFps);
+
+    return kCVReturnSuccess;
+}
+
+bool DisplayLinkVsyncSource::initialize(SDL_Window*, int displayFps)
+{
+    m_DisplayFps = displayFps;
+
+    CVDisplayLinkCreateWithActiveCGDisplays(&m_DisplayLink);
+    CVDisplayLinkSetOutputCallback(m_DisplayLink, displayLinkOutputCallback, this);
+    CVDisplayLinkStart(m_DisplayLink);
+    return true;
+}

app/streaming/video/ffmpeg-renderers/pacer/displaylinkvsyncsource.h

@@ -0,0 +1,31 @@
+#pragma once
+
+#include <CoreVideo/CoreVideo.h>
+
+#include "pacer.h"
+
+class DisplayLinkVsyncSource : public IVsyncSource
+{
+public:
+    DisplayLinkVsyncSource(Pacer* pacer);
+
+    virtual ~DisplayLinkVsyncSource();
+
+    virtual bool initialize(SDL_Window* window, int displayFps);
+
+private:
+    static
+    CVReturn
+    displayLinkOutputCallback(
+        CVDisplayLinkRef,
+        const CVTimeStamp* now,
+        const CVTimeStamp* /* vsyncTime */,
+        CVOptionFlags,
+        CVOptionFlags*,
+        void *displayLinkContext);
+
+    Pacer* m_Pacer;
+    CVDisplayLinkRef m_DisplayLink;
+    int m_DisplayFps;
+};
+

app/streaming/video/ffmpeg-renderers/pacer/dxvsyncsource.cpp

@@ -0,0 +1,168 @@
+#include "dxvsyncsource.h"
+
+// Useful references:
+// https://bugs.chromium.org/p/chromium/issues/detail?id=467617
+// https://chromium.googlesource.com/chromium/src.git/+/c564f2fe339b2b2abb0c8773c90c83215670ea71/gpu/ipc/service/gpu_vsync_provider_win.cc
+
+DxVsyncSource::DxVsyncSource(Pacer* pacer) :
+    m_Pacer(pacer),
+    m_Thread(nullptr),
+    m_Gdi32Handle(nullptr)
+{
+    SDL_AtomicSet(&m_Stopping, 0);
+}
+
+DxVsyncSource::~DxVsyncSource()
+{
+    if (m_Thread != nullptr) {
+        SDL_AtomicSet(&m_Stopping, 1);
+        SDL_WaitThread(m_Thread, nullptr);
+    }
+
+    if (m_Gdi32Handle != nullptr) {
+        FreeLibrary(m_Gdi32Handle);
+    }
+}
+
+bool DxVsyncSource::initialize(SDL_Window* window, int displayFps)
+{
+    m_DisplayFps = displayFps;
+
+    m_Gdi32Handle = LoadLibraryA("gdi32.dll");
+    if (m_Gdi32Handle == nullptr) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Failed to load gdi32.dll: %d",
+                     GetLastError());
+        return false;
+    }
+
+    m_D3DKMTOpenAdapterFromHdc = (PFND3DKMTOPENADAPTERFROMHDC)GetProcAddress(m_Gdi32Handle, "D3DKMTOpenAdapterFromHdc");
+    m_D3DKMTCloseAdapter = (PFND3DKMTCLOSEADAPTER)GetProcAddress(m_Gdi32Handle, "D3DKMTCloseAdapter");
+    m_D3DKMTWaitForVerticalBlankEvent = (PFND3DKMTWAITFORVERTICALBLANKEVENT)GetProcAddress(m_Gdi32Handle, "D3DKMTWaitForVerticalBlankEvent");
+
+    if (m_D3DKMTOpenAdapterFromHdc == nullptr ||
+            m_D3DKMTCloseAdapter == nullptr ||
+            m_D3DKMTWaitForVerticalBlankEvent == nullptr) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Missing required function in gdi32.dll");
+        return false;
+    }
+
+    SDL_SysWMinfo info;
+
+    SDL_VERSION(&info.version);
+
+    if (!SDL_GetWindowWMInfo(window, &info)) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "SDL_GetWindowWMInfo() failed: %s",
+                     SDL_GetError());
+        return false;
+    }
+
+    m_Window = info.info.win.window;
+
+    m_Thread = SDL_CreateThread(vsyncThread, "DX Vsync Thread", this);
+    if (m_Thread == nullptr) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Unable to create DX V-sync thread: %s",
+                     SDL_GetError());
+        return false;
+    }
+
+    return true;
+}
+
+int DxVsyncSource::vsyncThread(void* context)
+{
+    DxVsyncSource* me = reinterpret_cast<DxVsyncSource*>(context);
+
+    SDL_SetThreadPriority(SDL_THREAD_PRIORITY_HIGH);
+
+    D3DKMT_OPENADAPTERFROMHDC openAdapterParams = {};
+    HMONITOR lastMonitor = nullptr;
+    DEVMODEA monitorMode;
+    monitorMode.dmSize = sizeof(monitorMode);
+
+    while (SDL_AtomicGet(&me->m_Stopping) == 0) {
+        D3DKMT_WAITFORVERTICALBLANKEVENT waitForVblankEventParams;
+        NTSTATUS status;
+
+        // If the monitor has changed from last time, open the new adapter
+        HMONITOR currentMonitor = MonitorFromWindow(me->m_Window, MONITOR_DEFAULTTONEAREST);
+        if (currentMonitor != lastMonitor) {
+            MONITORINFOEXA monitorInfo = {};
+            monitorInfo.cbSize = sizeof(monitorInfo);
+            if (!GetMonitorInfoA(currentMonitor, &monitorInfo)) {
+                SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                             "GetMonitorInfo() failed: %d",
+                             GetLastError());
+                SDL_Delay(10);
+                continue;
+            }
+
+            if (!EnumDisplaySettingsA(monitorInfo.szDevice, ENUM_CURRENT_SETTINGS, &monitorMode)) {
+                SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                             "EnumDisplaySettings() failed: %d",
+                             GetLastError());
+                SDL_Delay(10);
+                continue;
+            }
+
+            SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                        "Monitor changed: %s %d Hz",
+                        monitorInfo.szDevice,
+                        monitorMode.dmDisplayFrequency);
+
+            if (openAdapterParams.hAdapter != 0) {
+                D3DKMT_CLOSEADAPTER closeAdapterParams = {};
+                closeAdapterParams.hAdapter = openAdapterParams.hAdapter;
+                me->m_D3DKMTCloseAdapter(&closeAdapterParams);
+            }
+
+            openAdapterParams.hDc = CreateDCA(nullptr, monitorInfo.szDevice, nullptr, nullptr);
+            if (!openAdapterParams.hDc) {
+                SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                             "CreateDC() failed: %d",
+                             GetLastError());
+                SDL_Delay(10);
+                continue;
+            }
+
+            status = me->m_D3DKMTOpenAdapterFromHdc(&openAdapterParams);
+            DeleteDC(openAdapterParams.hDc);
+
+            if (status != STATUS_SUCCESS) {
+                SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                             "D3DKMTOpenAdapterFromHdc() failed: %x",
+                             status);
+                SDL_Delay(10);
+                continue;
+            }
+
+            lastMonitor = currentMonitor;
+        }
+
+        waitForVblankEventParams.hAdapter = openAdapterParams.hAdapter;
+        waitForVblankEventParams.hDevice = 0;
+        waitForVblankEventParams.VidPnSourceId = openAdapterParams.VidPnSourceId;
+
+        status = me->m_D3DKMTWaitForVerticalBlankEvent(&waitForVblankEventParams);
+        if (status != STATUS_SUCCESS) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "D3DKMTWaitForVerticalBlankEvent() failed: %x",
+                         status);
+            SDL_Delay(10);
+            continue;
+        }
+
+        me->m_Pacer->vsyncCallback(1000 / me->m_DisplayFps);
+    }
+
+    if (openAdapterParams.hAdapter != 0) {
+        D3DKMT_CLOSEADAPTER closeAdapterParams = {};
+        closeAdapterParams.hAdapter = openAdapterParams.hAdapter;
+        me->m_D3DKMTCloseAdapter(&closeAdapterParams);
+    }
+
+    return 0;
+}

app/streaming/video/ffmpeg-renderers/pacer/dxvsyncsource.h

@@ -0,0 +1,53 @@
+#pragma once
+
+#include "pacer.h"
+
+#include <SDL_syswm.h>
+
+// from <D3dkmthk.h>
+typedef LONG NTSTATUS;
+typedef UINT D3DKMT_HANDLE;
+typedef UINT D3DDDI_VIDEO_PRESENT_SOURCE_ID;
+#define STATUS_SUCCESS ((NTSTATUS)0x00000000L)
+#define STATUS_GRAPHICS_PRESENT_OCCLUDED ((NTSTATUS)0xC01E0006L)
+typedef struct _D3DKMT_OPENADAPTERFROMHDC {
+  HDC hDc;
+  D3DKMT_HANDLE hAdapter;
+  LUID AdapterLuid;
+  D3DDDI_VIDEO_PRESENT_SOURCE_ID VidPnSourceId;
+} D3DKMT_OPENADAPTERFROMHDC;
+typedef struct _D3DKMT_CLOSEADAPTER {
+  D3DKMT_HANDLE hAdapter;
+} D3DKMT_CLOSEADAPTER;
+typedef struct _D3DKMT_WAITFORVERTICALBLANKEVENT {
+  D3DKMT_HANDLE hAdapter;
+  D3DKMT_HANDLE hDevice;
+  D3DDDI_VIDEO_PRESENT_SOURCE_ID VidPnSourceId;
+} D3DKMT_WAITFORVERTICALBLANKEVENT;
+typedef NTSTATUS(APIENTRY* PFND3DKMTOPENADAPTERFROMHDC)(D3DKMT_OPENADAPTERFROMHDC*);
+typedef NTSTATUS(APIENTRY* PFND3DKMTCLOSEADAPTER)(D3DKMT_CLOSEADAPTER*);
+typedef NTSTATUS(APIENTRY* PFND3DKMTWAITFORVERTICALBLANKEVENT)(D3DKMT_WAITFORVERTICALBLANKEVENT*);
+
+class DxVsyncSource : public IVsyncSource
+{
+public:
+    DxVsyncSource(Pacer* pacer);
+
+    virtual ~DxVsyncSource();
+
+    virtual bool initialize(SDL_Window* window, int displayFps);
+
+private:
+    static int vsyncThread(void* context);
+
+    Pacer* m_Pacer;
+    SDL_Thread* m_Thread;
+    SDL_atomic_t m_Stopping;
+    HMODULE m_Gdi32Handle;
+    HWND m_Window;
+    int m_DisplayFps;
+
+    PFND3DKMTOPENADAPTERFROMHDC m_D3DKMTOpenAdapterFromHdc;
+    PFND3DKMTCLOSEADAPTER m_D3DKMTCloseAdapter;
+    PFND3DKMTWAITFORVERTICALBLANKEVENT m_D3DKMTWaitForVerticalBlankEvent;
+};

app/streaming/video/ffmpeg-renderers/pacer/nullthreadedvsyncsource.cpp

@@ -0,0 +1,43 @@
+#include "nullthreadedvsyncsource.h"
+
+NullThreadedVsyncSource::NullThreadedVsyncSource(Pacer* pacer) :
+    m_Pacer(pacer),
+    m_Thread(nullptr)
+{
+    SDL_AtomicSet(&m_Stopping, 0);
+}
+
+NullThreadedVsyncSource::~NullThreadedVsyncSource()
+{
+    if (m_Thread != nullptr) {
+        SDL_AtomicSet(&m_Stopping, 1);
+        SDL_WaitThread(m_Thread, nullptr);
+    }
+}
+
+bool NullThreadedVsyncSource::initialize(SDL_Window*, int displayFps)
+{
+    m_DisplayFps = displayFps;
+    m_Thread = SDL_CreateThread(vsyncThread, "Null Vsync Thread", this);
+    if (m_Thread == nullptr) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Unable to create DX V-sync thread: %s",
+                     SDL_GetError());
+        return false;
+    }
+
+    return true;
+}
+
+int NullThreadedVsyncSource::vsyncThread(void* context)
+{
+    NullThreadedVsyncSource* me = reinterpret_cast<NullThreadedVsyncSource*>(context);
+
+    SDL_SetThreadPriority(SDL_THREAD_PRIORITY_HIGH);
+
+    while (SDL_AtomicGet(&me->m_Stopping) == 0) {
+        me->m_Pacer->vsyncCallback(1000 / me->m_DisplayFps);
+    }
+
+    return 0;
+}

app/streaming/video/ffmpeg-renderers/pacer/nullthreadedvsyncsource.h

@@ -0,0 +1,21 @@
+#pragma once
+
+#include "pacer.h"
+
+class NullThreadedVsyncSource : public IVsyncSource
+{
+public:
+    NullThreadedVsyncSource(Pacer* pacer);
+
+    virtual ~NullThreadedVsyncSource();
+
+    virtual bool initialize(SDL_Window* window, int displayFps);
+
+private:
+    static int vsyncThread(void* context);
+
+    Pacer* m_Pacer;
+    SDL_Thread* m_Thread;
+    SDL_atomic_t m_Stopping;
+    int m_DisplayFps;
+};

app/streaming/video/ffmpeg-renderers/pacer/pacer.cpp

@@ -0,0 +1,189 @@
+#include "pacer.h"
+
+#include "nullthreadedvsyncsource.h"
+
+#ifdef Q_OS_DARWIN
+#include "displaylinkvsyncsource.h"
+#endif
+
+#ifdef Q_OS_WIN32
+#include "dxvsyncsource.h"
+#endif
+
+#define FRAME_HISTORY_ENTRIES 8
+
+// We may be woken up slightly late so don't go all the way
+// up to the next V-sync since we may accidentally step into
+// the next V-sync period. It also takes some amount of time
+// to do the render itself, so we can't render right before
+// V-sync happens.
+#define TIMER_SLACK_MS 3
+
+Pacer::Pacer(IFFmpegRenderer* renderer) :
+    m_FrameQueueLock(0),
+    m_VsyncSource(nullptr),
+    m_VsyncRenderer(renderer),
+    m_MaxVideoFps(0),
+    m_DisplayFps(0)
+{
+
+}
+
+Pacer::~Pacer()
+{
+    // Stop V-sync callbacks
+    delete m_VsyncSource;
+    m_VsyncSource = nullptr;
+
+    while (!m_FrameQueue.isEmpty()) {
+        AVFrame* frame = m_FrameQueue.dequeue();
+        av_frame_free(&frame);
+    }
+}
+
+// Called in an arbitrary thread by the IVsyncSource on V-sync
+// or an event synchronized with V-sync
+void Pacer::vsyncCallback(int timeUntilNextVsyncMillis)
+{
+    // Make sure initialize() has been called
+    SDL_assert(m_MaxVideoFps != 0);
+
+    SDL_assert(timeUntilNextVsyncMillis >= TIMER_SLACK_MS);
+
+    Uint32 vsyncCallbackStartTime = SDL_GetTicks();
+
+    SDL_AtomicLock(&m_FrameQueueLock);
+
+    // If the queue length history entries are large, be strict
+    // about dropping excess frames.
+    int frameDropTarget = 1;
+
+    // If we may get more frames per second than we can display, use
+    // frame history to drop frames only if consistently above the
+    // one queued frame mark.
+    if (m_MaxVideoFps >= m_DisplayFps) {
+        for (int i = 0; i < m_FrameQueueHistory.count(); i++) {
+            if (m_FrameQueueHistory[i] <= 1) {
+                // Be lenient as long as the queue length
+                // resolves before the end of frame history
+                frameDropTarget = 3;
+                break;
+            }
+        }
+
+        if (m_FrameQueueHistory.count() == FRAME_HISTORY_ENTRIES) {
+            m_FrameQueueHistory.dequeue();
+        }
+
+        m_FrameQueueHistory.enqueue(m_FrameQueue.count());
+    }
+
+    // Catch up if we're several frames ahead
+    while (m_FrameQueue.count() > frameDropTarget) {
+        AVFrame* frame = m_FrameQueue.dequeue();
+        av_frame_free(&frame);
+    }
+
+
+    if (m_FrameQueue.isEmpty()) {
+        SDL_AtomicUnlock(&m_FrameQueueLock);
+
+        while (!SDL_TICKS_PASSED(SDL_GetTicks(),
+                                 vsyncCallbackStartTime + timeUntilNextVsyncMillis - TIMER_SLACK_MS)) {
+            SDL_Delay(1);
+
+            SDL_AtomicLock(&m_FrameQueueLock);
+            if (!m_FrameQueue.isEmpty()) {
+                // Don't release the lock
+                goto RenderNextFrame;
+            }
+            SDL_AtomicUnlock(&m_FrameQueueLock);
+        }
+
+        // Nothing to render at this time
+        return;
+    }
+
+RenderNextFrame:
+    // Grab the first frame
+    AVFrame* frame = m_FrameQueue.dequeue();
+    SDL_AtomicUnlock(&m_FrameQueueLock);
+
+    // Render it
+    m_VsyncRenderer->renderFrameAtVsync(frame);
+
+    // Free the frame
+    av_frame_free(&frame);
+}
+
+bool Pacer::initialize(SDL_Window* window, int maxVideoFps, bool enableVsync)
+{
+    m_MaxVideoFps = maxVideoFps;
+    m_EnableVsync = enableVsync;
+
+    int displayIndex = SDL_GetWindowDisplayIndex(window);
+    if (displayIndex < 0) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Failed to get current display: %s",
+                     SDL_GetError());
+
+        // Assume display 0 if it fails
+        displayIndex = 0;
+    }
+
+    SDL_DisplayMode mode;
+    if (SDL_GetCurrentDisplayMode(displayIndex, &mode) == 0) {
+        // May be zero if undefined
+        m_DisplayFps = mode.refresh_rate;
+    }
+
+    if (m_EnableVsync) {
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "Frame pacing in tear-free mode: target %d Hz with %d FPS stream",
+                    m_DisplayFps, m_MaxVideoFps);
+
+    #if defined(Q_OS_DARWIN)
+        m_VsyncSource = new DisplayLinkVsyncSource(this);
+    #elif defined(Q_OS_WIN32)
+        m_VsyncSource = new DxVsyncSource(this);
+    #else
+        // Platforms without a VsyncSource will just render frames
+        // immediately like they used to.
+    #endif
+
+        if (m_VsyncSource != nullptr && !m_VsyncSource->initialize(window, m_DisplayFps)) {
+            return false;
+        }
+    }
+    else {
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "Minimal latency tearing mode: target %d Hz with %d FPS stream",
+                    m_DisplayFps, m_MaxVideoFps);
+    }
+
+    return true;
+}
+
+void Pacer::submitFrame(AVFrame* frame)
+{
+    // Make sure initialize() has been called
+    SDL_assert(m_MaxVideoFps != 0);
+
+    // Queue the frame until the V-sync callback if
+    // we have a V-sync source, otherwise deliver it
+    // immediately and hope for the best.
+    if (isUsingFrameQueue()) {
+        SDL_AtomicLock(&m_FrameQueueLock);
+        m_FrameQueue.enqueue(frame);
+        SDL_AtomicUnlock(&m_FrameQueueLock);
+    }
+    else {
+        m_VsyncRenderer->renderFrameAtVsync(frame);
+        av_frame_free(&frame);
+    }
+}
+
+bool Pacer::isUsingFrameQueue()
+{
+    return m_VsyncSource != nullptr;
+}

app/streaming/video/ffmpeg-renderers/pacer/pacer.h

@@ -0,0 +1,38 @@
+#pragma once
+
+#include "../renderer.h"
+
+#include <QQueue>
+
+class IVsyncSource {
+public:
+    virtual ~IVsyncSource() {}
+    virtual bool initialize(SDL_Window* window, int displayFps) = 0;
+};
+
+class Pacer
+{
+public:
+    Pacer(IFFmpegRenderer* renderer);
+
+    ~Pacer();
+
+    void submitFrame(AVFrame* frame);
+
+    bool initialize(SDL_Window* window, int maxVideoFps, bool enableVsync);
+
+    void vsyncCallback(int timeUntilNextVsyncMillis);
+
+    bool isUsingFrameQueue();
+
+private:
+    QQueue<AVFrame*> m_FrameQueue;
+    QQueue<int> m_FrameQueueHistory;
+    SDL_SpinLock m_FrameQueueLock;
+
+    IVsyncSource* m_VsyncSource;
+    IFFmpegRenderer* m_VsyncRenderer;
+    int m_MaxVideoFps;
+    int m_DisplayFps;
+    bool m_EnableVsync;
+};

app/streaming/video/ffmpeg-renderers/renderer.h

@@ -12,9 +12,12 @@ public:
     virtual bool initialize(SDL_Window* window,
                             int videoFormat,
                             int width,
-                            int height) = 0;
+                            int height,
+                            int maxFps,
+                            bool enableVsync) = 0;
     virtual bool prepareDecoderContext(AVCodecContext* context) = 0;
-    virtual void renderFrame(AVFrame* frame) = 0;
+    virtual void renderFrameAtVsync(AVFrame* frame) = 0;
+    virtual bool needsTestFrame() = 0;
 };
 
 class SdlRenderer : public IFFmpegRenderer {
@@ -24,9 +27,12 @@ public:
     virtual bool initialize(SDL_Window* window,
                             int videoFormat,
                             int width,
-                            int height);
+                            int height,
+                            int maxFps,
+                            bool enableVsync);
     virtual bool prepareDecoderContext(AVCodecContext* context);
-    virtual void renderFrame(AVFrame* frame);
+    virtual void renderFrameAtVsync(AVFrame* frame);
+    virtual bool needsTestFrame();
 
 private:
     SDL_Renderer* m_Renderer;

app/streaming/video/ffmpeg-renderers/sdl.cpp

@@ -23,23 +23,38 @@ SdlRenderer::~SdlRenderer()
 bool SdlRenderer::prepareDecoderContext(AVCodecContext*)
 {
     /* Nothing to do */
+
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Using SDL software renderer");
+
     return true;
 }
 
-bool SdlRenderer::initialize(SDL_Window* window,
+bool SdlRenderer::needsTestFrame()
-                             int videoFormat,
-                             int width,
-                             int height)
 {
-    // NOTE: HEVC currently uses only 1 slice regardless of what
+    // This renderer should always work
-    // we provide in CAPABILITY_SLICES_PER_FRAME(), so we should
-    // never use it for software decoding (unless common-c starts
-    // respecting it for HEVC).
-    if (videoFormat != VIDEO_FORMAT_H264) {
     return false;
 }
 
-    m_Renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
+bool SdlRenderer::initialize(SDL_Window* window,
+                             int,
+                             int width,
+                             int height,
+                             int,
+                             bool enableVsync)
+{
+    Uint32 rendererFlags = SDL_RENDERER_ACCELERATED;
+
+    if ((SDL_GetWindowFlags(window) & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN) {
+        // In full-screen exclusive mode, we enable V-sync if requested. For other modes, Windows and Mac
+        // have compositors that make rendering tear-free. Linux compositor varies by distro and user
+        // configuration but doesn't seem feasible to detect here.
+        if (enableVsync) {
+            rendererFlags |= SDL_RENDERER_PRESENTVSYNC;
+        }
+    }
+
+    m_Renderer = SDL_CreateRenderer(window, -1, rendererFlags);
     if (!m_Renderer) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                      "SDL_CreateRenderer() failed: %s",
@@ -71,7 +86,7 @@ bool SdlRenderer::initialize(SDL_Window* window,
     return true;
 }
 
-void SdlRenderer::renderFrame(AVFrame* frame)
+void SdlRenderer::renderFrameAtVsync(AVFrame* frame)
 {
     SDL_UpdateYUVTexture(m_Texture, nullptr,
                          frame->data[0],
@@ -81,9 +96,6 @@ void SdlRenderer::renderFrame(AVFrame* frame)
                          frame->data[2],
                          frame->linesize[2]);
 
-    // Done with the frame now
-    av_frame_free(&frame);
-
     SDL_RenderClear(m_Renderer);
     SDL_RenderCopy(m_Renderer, m_Texture, nullptr, nullptr);
     SDL_RenderPresent(m_Renderer);

app/streaming/video/ffmpeg-renderers/vaapi.cpp

@@ -1,13 +1,10 @@
 #include "vaapi.h"
-
+#include <streaming/streamutils.h>
-#include <dlfcn.h>
 
 #include <SDL_syswm.h>
 
 VAAPIRenderer::VAAPIRenderer()
-    : m_HwContext(nullptr),
+    : m_HwContext(nullptr)
-      m_X11VaLibHandle(nullptr),
-      m_vaPutSurface(nullptr)
 {
 
 }
@@ -15,16 +12,22 @@ VAAPIRenderer::VAAPIRenderer()
 VAAPIRenderer::~VAAPIRenderer()
 {
     if (m_HwContext != nullptr) {
-        av_buffer_unref(&m_HwContext);
+        AVHWDeviceContext* deviceContext = (AVHWDeviceContext*)m_HwContext->data;
-    }
+        AVVAAPIDeviceContext* vaDeviceContext = (AVVAAPIDeviceContext*)deviceContext->hwctx;
 
-    if (m_X11VaLibHandle != nullptr) {
+        // Hold onto this VADisplay since we'll need it to uninitialize VAAPI
-        dlclose(m_X11VaLibHandle);
+        VADisplay display = vaDeviceContext->display;
+
+        av_buffer_unref(&m_HwContext);
+
+        if (display) {
+            vaTerminate(display);
+        }
     }
 }
 
 bool
-VAAPIRenderer::initialize(SDL_Window* window, int, int width, int height)
+VAAPIRenderer::initialize(SDL_Window* window, int, int width, int height, int, bool)
 {
     int err;
     SDL_SysWMinfo info;
@@ -37,32 +40,54 @@ VAAPIRenderer::initialize(SDL_Window* window, int, int width, int height)
     SDL_VERSION(&info.version);
 
     if (!SDL_GetWindowWMInfo(window, &info)) {
-        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                      "SDL_GetWindowWMInfo() failed: %s",
                      SDL_GetError());
         return false;
     }
 
-    SDL_assert(info.subsystem == SDL_SYSWM_X11);
+    m_HwContext = av_hwdevice_ctx_alloc(AV_HWDEVICE_TYPE_VAAPI);
-
+    if (!m_HwContext) {
-    if (info.subsystem == SDL_SYSWM_X11) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-        m_XWindow = info.info.x11.window;
+                    "Failed to allocate VAAPI context");
-
-        m_X11VaLibHandle = dlopen("libva-x11.so", RTLD_LAZY);
-        if (!m_X11VaLibHandle) {
-            SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
-                        "dlopen(libva.so) failed: %s",
-                        dlerror());
         return false;
     }
 
-        m_vaPutSurface = (vaPutSurface_t)dlsym(m_X11VaLibHandle, "vaPutSurface");
+    AVHWDeviceContext* deviceContext = (AVHWDeviceContext*)m_HwContext->data;
+    AVVAAPIDeviceContext* vaDeviceContext = (AVVAAPIDeviceContext*)deviceContext->hwctx;
+
+    m_WindowSystem = info.subsystem;
+    if (info.subsystem == SDL_SYSWM_X11) {
+#ifdef HAVE_LIBVA_X11
+        m_XWindow = info.info.x11.window;
+        vaDeviceContext->display = vaGetDisplay(info.info.x11.display);
+        if (!vaDeviceContext->display) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "Unable to open X11 display for VAAPI");
+            return false;
+        }
+#else
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Moonlight not compiled with VAAPI X11 support!");
+        return false;
+#endif
     }
     else if (info.subsystem == SDL_SYSWM_WAYLAND) {
+#ifdef HAVE_LIBVA_WAYLAND
+        m_WaylandSurface = info.info.wl.surface;
+        m_WaylandDisplay = info.info.wl.display;
+        vaDeviceContext->display = vaGetDisplayWl(info.info.wl.display);
+        if (!vaDeviceContext->display) {
             SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                     "VAAPI backend does not currently support Wayland");
+                         "Unable to open Wayland display for VAAPI");
             return false;
         }
+#else
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Moonlight not compiled with VAAPI Wayland support!");
+        return false;
+#endif
+    }
     else {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                      "Unsupported VAAPI rendering subsystem: %d",
@@ -70,17 +95,37 @@ VAAPIRenderer::initialize(SDL_Window* window, int, int width, int height)
         return false;
     }
 
+    int major, minor;
+    VAStatus status;
+    status = vaInitialize(vaDeviceContext->display, &major, &minor);
+    if (status != VA_STATUS_SUCCESS) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Failed to initialize VAAPI: %d",
+                     status);
+        return false;
+    }
 
-    err = av_hwdevice_ctx_create(&m_HwContext,
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
-                                 AV_HWDEVICE_TYPE_VAAPI,
+                "Initialized VAAPI %d.%d",
-                                 nullptr, nullptr, 0);
+                major, minor);
+
+    // This will populate the driver_quirks
+    err = av_hwdevice_ctx_init(m_HwContext);
     if (err < 0) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                     "Failed to create VAAPI context: %d",
+                     "Failed to initialize VAAPI context: %d",
                      err);
         return false;
     }
 
+    // This quirk is set for the VDPAU wrapper which doesn't work with our VAAPI renderer
+    if (vaDeviceContext->driver_quirks & AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES) {
+        // Fail and let our VDPAU renderer pick this up
+        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                    "Avoiding VDPAU wrapper for VAAPI decoding");
+        return false;
+    }
+
     return true;
 }
 
@@ -88,45 +133,76 @@ bool
 VAAPIRenderer::prepareDecoderContext(AVCodecContext* context)
 {
     context->hw_device_ctx = av_buffer_ref(m_HwContext);
+
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Using VAAPI accelerated renderer on %s",
+                m_WindowSystem == SDL_SYSWM_X11 ? "X11" : "Wayland");
+
+    return true;
+}
+
+bool
+VAAPIRenderer::needsTestFrame()
+{
+    // We need a test frame to see if this VAAPI driver
+    // supports the profile used for streaming
     return true;
 }
 
 void
-VAAPIRenderer::renderFrame(AVFrame* frame)
+VAAPIRenderer::renderFrameAtVsync(AVFrame* frame)
 {
     VASurfaceID surface = (VASurfaceID)(uintptr_t)frame->data[3];
     AVHWDeviceContext* deviceContext = (AVHWDeviceContext*)m_HwContext->data;
     AVVAAPIDeviceContext* vaDeviceContext = (AVVAAPIDeviceContext*)deviceContext->hwctx;
 
-    // Center in frame and preserve aspect ratio
+    SDL_Rect src, dst;
-    int x, y, width, height;
+    src.x = src.y = 0;
-    double srcAspectRatio = (double)m_VideoWidth / (double)m_VideoHeight;
+    src.w = m_VideoWidth;
-    double dstAspectRatio = (double)m_DisplayWidth / (double)m_DisplayHeight;
+    src.h = m_VideoHeight;
-    if (dstAspectRatio < srcAspectRatio) {
+    dst.x = dst.y = 0;
-        // Greater height per width
+    dst.w = m_DisplayWidth;
-        int drawHeight = (int)(m_DisplayWidth / srcAspectRatio);
+    dst.h = m_DisplayHeight;
-        y = (m_DisplayHeight - drawHeight) / 2;
-        height = drawHeight;
-        x = 0;
-        width = m_DisplayWidth;
-    }
-    else {
-        // Greater width per height
-        int drawWidth = (int)(m_DisplayHeight * srcAspectRatio);
-        y = 0;
-        height = m_DisplayHeight;
-        x = (m_DisplayWidth - drawWidth) / 2;
-        width = drawWidth;
-    }
 
-    m_vaPutSurface(vaDeviceContext->display,
+    StreamUtils::scaleSourceToDestinationSurface(&src, &dst);
+
+    if (m_WindowSystem == SDL_SYSWM_X11) {
+#ifdef HAVE_LIBVA_X11
+        vaPutSurface(vaDeviceContext->display,
                      surface,
                      m_XWindow,
                      0, 0,
                      m_VideoWidth, m_VideoHeight,
-                   x, y,
+                     dst.x, dst.y,
-                   width, height,
+                     dst.w, dst.h,
                      NULL, 0, 0);
+#endif
+    }
+    else if (m_WindowSystem == SDL_SYSWM_WAYLAND) {
+#ifdef HAVE_LIBVA_WAYLAND
+        struct wl_buffer* buffer;
+        VAStatus status;
 
-    av_frame_free(&frame);
+        status = vaGetSurfaceBufferWl(vaDeviceContext->display,
+                                      surface,
+                                      VA_FRAME_PICTURE,
+                                      &buffer);
+        if (status == VA_STATUS_SUCCESS) {
+            wl_surface_attach(m_WaylandSurface, buffer, 0, 0);
+            wl_surface_damage(m_WaylandSurface, dst.x, dst.y, dst.w, dst.h);
+
+            wl_display_flush(m_WaylandDisplay);
+            wl_surface_commit(m_WaylandSurface);
+        }
+        else {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "vaGetSurfaceBufferWl failed(): %d",
+                         status);
+        }
+#endif
+    }
+    else {
+        // We don't accept anything else in initialize().
+        SDL_assert(false);
+    }
 }

app/streaming/video/ffmpeg-renderers/vaapi.h

@@ -2,49 +2,59 @@
 
 #include "renderer.h"
 
+// Avoid X11 if SDL was built without it
+#ifndef SDL_VIDEO_DRIVER_X11
+#warning Unable to use libva-x11 without SDL support
+#undef HAVE_LIBVA_X11
+#endif
+
+// Avoid Wayland if SDL was built without it
+#ifndef SDL_VIDEO_DRIVER_WAYLAND
+#warning Unable to use libva-wayland without SDL support
+#undef HAVE_LIBVA_WAYLAND
+#endif
+
 extern "C" {
 #include <va/va.h>
+#ifdef HAVE_LIBVA_X11
 #include <va/va_x11.h>
+#endif
+#ifdef HAVE_LIBVA_WAYLAND
+#include <va/va_wayland.h>
+#endif
 #include <libavutil/hwcontext_vaapi.h>
 }
 
 class VAAPIRenderer : public IFFmpegRenderer
 {
-    typedef VAStatus (*vaPutSurface_t)(
-        VADisplay dpy,
-        VASurfaceID surface,
-        Drawable draw, /* X Drawable */
-        short srcx,
-        short srcy,
-        unsigned short srcw,
-        unsigned short srch,
-        short destx,
-        short desty,
-        unsigned short destw,
-        unsigned short desth,
-        VARectangle *cliprects,
-        unsigned int number_cliprects,
-        unsigned int flags
-    );
-
 public:
     VAAPIRenderer();
     virtual ~VAAPIRenderer();
     virtual bool initialize(SDL_Window* window,
                             int videoFormat,
                             int width,
-                            int height);
+                            int height,
+                            int maxFps,
+                            bool enableVsync);
     virtual bool prepareDecoderContext(AVCodecContext* context);
-    virtual void renderFrame(AVFrame* frame);
+    virtual void renderFrameAtVsync(AVFrame* frame);
+    virtual bool needsTestFrame();
 
 private:
-    Window m_XWindow;
+    int m_WindowSystem;
     AVBufferRef* m_HwContext;
-    void* m_X11VaLibHandle;
+
-    vaPutSurface_t m_vaPutSurface;
+#ifdef HAVE_LIBVA_X11
+    Window m_XWindow;
+#endif
+
+#ifdef HAVE_LIBVA_WAYLAND
+    struct wl_surface* m_WaylandSurface;
+    struct wl_display* m_WaylandDisplay;
+#endif
+
     int m_VideoWidth;
     int m_VideoHeight;
     int m_DisplayWidth;
     int m_DisplayHeight;
 };
-

app/streaming/video/ffmpeg-renderers/vdpau.cpp

@@ -0,0 +1,336 @@
+#include "vdpau.h"
+#include <streaming/streamutils.h>
+
+#include <SDL_syswm.h>
+
+#define BAIL_ON_FAIL(status, something) if ((status) != VDP_STATUS_OK) { \
+                                            SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, \
+                                                        #something " failed: %d", (status)); \
+                                            return false; \
+                                        }
+
+#define GET_PROC_ADDRESS(id, func) status = vdpauCtx->get_proc_address(m_Device, id, (void**)func); \
+                                   BAIL_ON_FAIL(status, id)
+
+const VdpRGBAFormat VDPAURenderer::k_OutputFormats[] = {
+    VDP_RGBA_FORMAT_B8G8R8A8,
+    VDP_RGBA_FORMAT_R8G8B8A8
+};
+
+VDPAURenderer::VDPAURenderer()
+    : m_HwContext(nullptr),
+      m_PresentationQueueTarget(0),
+      m_PresentationQueue(0),
+      m_VideoMixer(0),
+      m_NextSurfaceIndex(0)
+{
+    SDL_zero(m_OutputSurface);
+}
+
+VDPAURenderer::~VDPAURenderer()
+{
+    if (m_PresentationQueue != 0) {
+        m_VdpPresentationQueueDestroy(m_PresentationQueue);
+    }
+
+    if (m_VideoMixer != 0) {
+        m_VdpVideoMixerDestroy(m_VideoMixer);
+    }
+
+    if (m_PresentationQueueTarget != 0) {
+        m_VdpPresentationQueueTargetDestroy(m_PresentationQueueTarget);
+    }
+
+    for (int i = 0; i < OUTPUT_SURFACE_COUNT; i++) {
+        if (m_OutputSurface[i] != 0) {
+            m_VdpOutputSurfaceDestroy(m_OutputSurface[i]);
+        }
+    }
+
+    // This must be done last as it frees VDPAU context required to call
+    // the functions above.
+    if (m_HwContext != nullptr) {
+        av_buffer_unref(&m_HwContext);
+    }
+}
+
+bool VDPAURenderer::initialize(SDL_Window* window, int, int width, int height, int, bool)
+{
+    int err;
+    VdpStatus status;
+    SDL_SysWMinfo info;
+
+    m_VideoWidth = width;
+    m_VideoHeight = height;
+
+    err = av_hwdevice_ctx_create(&m_HwContext,
+                                 AV_HWDEVICE_TYPE_VDPAU,
+                                 nullptr, nullptr, 0);
+    if (err < 0) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Failed to create VDPAU context: %d",
+                     err);
+        return false;
+    }
+
+    AVHWDeviceContext* devCtx = (AVHWDeviceContext*)m_HwContext->data;
+    AVVDPAUDeviceContext* vdpauCtx = (AVVDPAUDeviceContext*)devCtx->hwctx;
+    m_Device = vdpauCtx->device;
+
+    GET_PROC_ADDRESS(VDP_FUNC_ID_GET_ERROR_STRING, &m_VdpGetErrorString);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_TARGET_DESTROY, &m_VdpPresentationQueueTargetDestroy);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_VIDEO_MIXER_CREATE, &m_VdpVideoMixerCreate);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_VIDEO_MIXER_DESTROY, &m_VdpVideoMixerDestroy);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_VIDEO_MIXER_RENDER, &m_VdpVideoMixerRender);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_CREATE, &m_VdpPresentationQueueCreate);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_DESTROY, &m_VdpPresentationQueueDestroy);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_DISPLAY, &m_VdpPresentationQueueDisplay);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_SET_BACKGROUND_COLOR, &m_VdpPresentationQueueSetBackgroundColor);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_BLOCK_UNTIL_SURFACE_IDLE, &m_VdpPresentationQueueBlockUntilSurfaceIdle);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_OUTPUT_SURFACE_CREATE, &m_VdpOutputSurfaceCreate);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_OUTPUT_SURFACE_DESTROY, &m_VdpOutputSurfaceDestroy);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_OUTPUT_SURFACE_QUERY_CAPABILITIES, &m_VdpOutputSurfaceQueryCapabilities);
+    GET_PROC_ADDRESS(VDP_FUNC_ID_VIDEO_SURFACE_GET_PARAMETERS, &m_VdpVideoSurfaceGetParameters);
+
+    SDL_GetWindowSize(window, (int*)&m_DisplayWidth, (int*)&m_DisplayHeight);
+
+    SDL_VERSION(&info.version);
+
+    if (!SDL_GetWindowWMInfo(window, &info)) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "SDL_GetWindowWMInfo() failed: %s",
+                     SDL_GetError());
+        return false;
+    }
+
+    SDL_assert(info.subsystem == SDL_SYSWM_X11);
+
+    if (info.subsystem == SDL_SYSWM_X11) {
+        GET_PROC_ADDRESS(VDP_FUNC_ID_PRESENTATION_QUEUE_TARGET_CREATE_X11,
+                         &m_VdpPresentationQueueTargetCreateX11);
+        status = m_VdpPresentationQueueTargetCreateX11(m_Device,
+                                                       info.info.x11.window,
+                                                       &m_PresentationQueueTarget);
+        if (status != VDP_STATUS_OK) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "VdpPresentationQueueTargetCreateX11() failed: %s",
+                         m_VdpGetErrorString(status));
+            return false;
+        }
+    }
+    else if (info.subsystem == SDL_SYSWM_WAYLAND) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "VDPAU backend does not currently support Wayland");
+        return false;
+    }
+    else {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Unsupported VDPAU rendering subsystem: %d",
+                     info.subsystem);
+        return false;
+    }
+
+    // Try our available output formats to find something the GPU supports
+    bool foundFormat = false;
+    for (int i = 0; i < OUTPUT_SURFACE_FORMAT_COUNT; i++) {
+        VdpBool supported;
+        uint32_t maxWidth, maxHeight;
+        status = m_VdpOutputSurfaceQueryCapabilities(m_Device, k_OutputFormats[i],
+                                                     &supported, &maxWidth, &maxHeight);
+        if (status != VDP_STATUS_OK) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "VdpOutputSurfaceQueryCapabilities() failed: %s",
+                         m_VdpGetErrorString(status));
+            return false;
+        }
+
+        if (supported) {
+            if (m_DisplayWidth <= maxWidth && m_DisplayHeight <= maxHeight) {
+                m_OutputSurfaceFormat = k_OutputFormats[i];
+                foundFormat = true;
+                break;
+            }
+            else  {
+                SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                            "Display size not within capabilities %dx%d vs %dx%d",
+                            m_DisplayWidth, m_DisplayWidth,
+                            maxWidth, maxHeight);
+            }
+        }
+    }
+
+    if (!foundFormat) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "No compatible output surface format found!");
+        return false;
+    }
+
+    // Create the output surfaces
+    for (int i = 0; i < OUTPUT_SURFACE_COUNT; i++) {
+        // It seems there's some lazy freeing going on or something in VDPAU
+        // because we can get VDP_STATUS_RESOURCES, then wait a bit and it'll
+        // complete without a problem.
+        int tries = 1;
+        do {
+            status = m_VdpOutputSurfaceCreate(m_Device, m_OutputSurfaceFormat,
+                                              m_DisplayWidth, m_DisplayHeight,
+                                              &m_OutputSurface[i]);
+            if (status != VDP_STATUS_OK) {
+                SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                            "VdpOutputSurfaceCreate() try #%d: %s",
+                            tries,
+                            m_VdpGetErrorString(status));
+                SDL_Delay(250);
+            }
+        } while (status == VDP_STATUS_RESOURCES && ++tries <= 10);
+
+        if (status != VDP_STATUS_OK) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "VdpOutputSurfaceCreate() failed: %s",
+                         m_VdpGetErrorString(status));
+            return false;
+        }
+    }
+
+    status = m_VdpPresentationQueueCreate(m_Device, m_PresentationQueueTarget,
+                                          &m_PresentationQueue);
+    if (status != VDP_STATUS_OK) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "VdpPresentationQueueCreate() failed: %s",
+                     m_VdpGetErrorString(status));
+        return false;
+    }
+
+    // Set the background to opaque black
+    VdpColor color = {0.0, 0.0, 0.0, 1.0};
+    m_VdpPresentationQueueSetBackgroundColor(m_PresentationQueue, &color);
+
+    return true;
+}
+
+bool VDPAURenderer::prepareDecoderContext(AVCodecContext* context)
+{
+    context->hw_device_ctx = av_buffer_ref(m_HwContext);
+
+    // Allow HEVC usage on VDPAU. This was disabled by FFmpeg due to
+    // GL interop issues, but we use VDPAU for rendering so it's no issue.
+    // https://github.com/FFmpeg/FFmpeg/commit/64ecb78b7179cab2dbdf835463104679dbb7c895
+    context->hwaccel_flags |= AV_HWACCEL_FLAG_ALLOW_PROFILE_MISMATCH;
+
+    SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                "Using VDPAU accelerated renderer");
+
+    return true;
+}
+
+bool VDPAURenderer::needsTestFrame()
+{
+    // We need a test frame to see if this VDPAU driver
+    // supports the profile used for streaming
+    return true;
+}
+
+void VDPAURenderer::renderFrameAtVsync(AVFrame* frame)
+{
+    VdpStatus status;
+    VdpVideoSurface videoSurface = (VdpVideoSurface)(uintptr_t)frame->data[3];
+
+    // This is safe without locking because this is always called on the main thread
+    VdpOutputSurface chosenSurface = m_OutputSurface[m_NextSurfaceIndex];
+    m_NextSurfaceIndex = (m_NextSurfaceIndex + 1) % OUTPUT_SURFACE_COUNT;
+
+    // We need to create the mixer on the fly, because we don't know the dimensions
+    // of our video surfaces in advance of decoding
+    if (m_VideoMixer == 0) {
+        VdpChromaType videoSurfaceChroma;
+        uint32_t videoSurfaceWidth, videoSurfaceHeight;
+        status = m_VdpVideoSurfaceGetParameters(videoSurface, &videoSurfaceChroma,
+                                                &videoSurfaceWidth, &videoSurfaceHeight);
+        if (status != VDP_STATUS_OK) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "VdpVideoSurfaceGetParameters() failed: %s",
+                         m_VdpGetErrorString(status));
+            return;
+        }
+
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "VDPAU surface size: %dx%d",
+                    videoSurfaceWidth, videoSurfaceHeight);
+
+    #define PARAM_COUNT 3
+        const VdpVideoMixerParameter params[PARAM_COUNT] = {
+            VDP_VIDEO_MIXER_PARAMETER_VIDEO_SURFACE_WIDTH,
+            VDP_VIDEO_MIXER_PARAMETER_VIDEO_SURFACE_HEIGHT,
+            VDP_VIDEO_MIXER_PARAMETER_CHROMA_TYPE,
+        };
+        const void* const paramValues[PARAM_COUNT] = {
+            &videoSurfaceWidth,
+            &videoSurfaceHeight,
+            &videoSurfaceChroma,
+        };
+
+        status = m_VdpVideoMixerCreate(m_Device, 0, nullptr,
+                                       PARAM_COUNT, params, paramValues,
+                                       &m_VideoMixer);
+        if (status != VDP_STATUS_OK) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "VdpVideoMixerCreate() failed: %s",
+                         m_VdpGetErrorString(status));
+            return;
+        }
+    }
+
+    // Wait for this frame to be off the screen
+    VdpTime pts;
+    m_VdpPresentationQueueBlockUntilSurfaceIdle(m_PresentationQueue, chosenSurface, &pts);
+
+    VdpRect sourceRect, outputRect;
+
+    SDL_Rect src, dst;
+    src.x = src.y = 0;
+    src.w = m_VideoWidth;
+    src.h = m_VideoHeight;
+    dst.x = dst.y = 0;
+    dst.w = m_DisplayWidth;
+    dst.h = m_DisplayHeight;
+
+    StreamUtils::scaleSourceToDestinationSurface(&src, &dst);
+
+    outputRect.x0 = dst.x;
+    outputRect.x1 = dst.x + dst.w;
+    outputRect.y0 = dst.y;
+    outputRect.y1 = dst.y + dst.h;
+
+    sourceRect.x0 = sourceRect.y0 = 0;
+    sourceRect.x1 = m_VideoWidth;
+    sourceRect.y1 = m_VideoHeight;
+
+    // Render the next frame into the output surface
+    status = m_VdpVideoMixerRender(m_VideoMixer,
+                                   VDP_INVALID_HANDLE, nullptr,
+                                   VDP_VIDEO_MIXER_PICTURE_STRUCTURE_FRAME,
+                                   0, nullptr,
+                                   videoSurface,
+                                   0, nullptr,
+                                   &sourceRect,
+                                   chosenSurface,
+                                   &outputRect,
+                                   nullptr,
+                                   0,
+                                   nullptr);
+    if (status != VDP_STATUS_OK) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "VdpVideoMixerRender() failed: %s",
+                     m_VdpGetErrorString(status));
+        return;
+    }
+
+    // Queue the frame for display immediately
+    status = m_VdpPresentationQueueDisplay(m_PresentationQueue, chosenSurface, 0, 0, 0);
+    if (status != VDP_STATUS_OK) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "VdpPresentationQueueDisplay() failed: %s",
+                     m_VdpGetErrorString(status));
+        return;
+    }
+}

app/streaming/video/ffmpeg-renderers/vdpau.h

@@ -0,0 +1,62 @@
+#pragma once
+
+#include "renderer.h"
+
+extern "C" {
+#include <vdpau/vdpau.h>
+#include <vdpau/vdpau_x11.h>
+#include <libavutil/hwcontext_vdpau.h>
+}
+
+class VDPAURenderer : public IFFmpegRenderer
+{
+public:
+    VDPAURenderer();
+    virtual ~VDPAURenderer();
+    virtual bool initialize(SDL_Window* window,
+                            int videoFormat,
+                            int width,
+                            int height,
+                            int maxFps,
+                            bool enableVsync);
+    virtual bool prepareDecoderContext(AVCodecContext* context);
+    virtual void renderFrameAtVsync(AVFrame* frame);
+    virtual bool needsTestFrame();
+
+private:
+    uint32_t m_VideoWidth, m_VideoHeight;
+    uint32_t m_DisplayWidth, m_DisplayHeight;
+    AVBufferRef* m_HwContext;
+    VdpPresentationQueueTarget m_PresentationQueueTarget;
+    VdpPresentationQueue m_PresentationQueue;
+    VdpVideoMixer m_VideoMixer;
+    VdpRGBAFormat m_OutputSurfaceFormat;
+    VdpDevice m_Device;
+
+#define OUTPUT_SURFACE_COUNT 3
+    VdpOutputSurface m_OutputSurface[OUTPUT_SURFACE_COUNT];
+    int m_NextSurfaceIndex;
+
+#define OUTPUT_SURFACE_FORMAT_COUNT 2
+    static const VdpRGBAFormat k_OutputFormats[OUTPUT_SURFACE_FORMAT_COUNT];
+
+    VdpGetErrorString* m_VdpGetErrorString;
+    VdpPresentationQueueTargetDestroy* m_VdpPresentationQueueTargetDestroy;
+    VdpVideoMixerCreate* m_VdpVideoMixerCreate;
+    VdpVideoMixerDestroy* m_VdpVideoMixerDestroy;
+    VdpVideoMixerRender* m_VdpVideoMixerRender;
+    VdpPresentationQueueCreate* m_VdpPresentationQueueCreate;
+    VdpPresentationQueueDestroy* m_VdpPresentationQueueDestroy;
+    VdpPresentationQueueDisplay* m_VdpPresentationQueueDisplay;
+    VdpPresentationQueueSetBackgroundColor* m_VdpPresentationQueueSetBackgroundColor;
+    VdpPresentationQueueBlockUntilSurfaceIdle* m_VdpPresentationQueueBlockUntilSurfaceIdle;
+    VdpOutputSurfaceCreate* m_VdpOutputSurfaceCreate;
+    VdpOutputSurfaceDestroy* m_VdpOutputSurfaceDestroy;
+    VdpOutputSurfaceQueryCapabilities* m_VdpOutputSurfaceQueryCapabilities;
+    VdpVideoSurfaceGetParameters* m_VdpVideoSurfaceGetParameters;
+
+    // X11 stuff
+    VdpPresentationQueueTargetCreateX11* m_VdpPresentationQueueTargetCreateX11;
+};
+
+

app/streaming/video/ffmpeg-renderers/vt.mm

@@ -2,19 +2,16 @@
 // libavutil both defining AVMediaType
 #define AVMediaType AVMediaType_FFmpeg
 #include "vt.h"
+#include "pacer/pacer.h"
 #undef AVMediaType
 
 #include <SDL_syswm.h>
 #include <Limelight.h>
 
-#include <QQueue>
+#include <mach/mach_time.h>
-
 #import <Cocoa/Cocoa.h>
 #import <VideoToolbox/VideoToolbox.h>
 #import <AVFoundation/AVFoundation.h>
-#import <CoreVideo/CoreVideo.h>
-
-#define FRAME_HISTORY_ENTRIES 8
 
 class VTRenderer : public IFFmpegRenderer
 {
@@ -23,9 +20,7 @@ public:
         : m_HwContext(nullptr),
           m_DisplayLayer(nullptr),
           m_FormatDesc(nullptr),
-          m_View(nullptr),
+          m_View(nullptr)
-          m_DisplayLink(nullptr),
-          m_FrameQueueLock(0)
     {
     }
 
@@ -39,63 +34,28 @@ public:
             CFRelease(m_FormatDesc);
         }
 
-        if (m_DisplayLink != nullptr) {
-            CVDisplayLinkStop(m_DisplayLink);
-            CVDisplayLinkRelease(m_DisplayLink);
-        }
-
-        while (!m_FrameQueue.isEmpty()) {
-            AVFrame* frame = m_FrameQueue.dequeue();
-            av_frame_free(&frame);
-        }
-
         if (m_View != nullptr) {
             [m_View removeFromSuperview];
         }
     }
 
-    void drawFrame(uint64_t vsyncTime)
+    // Caller frees frame after we return
+    virtual void renderFrameAtVsync(AVFrame* frame) override
     {
         OSStatus status;
+        CVPixelBufferRef pixBuf = reinterpret_cast<CVPixelBufferRef>(frame->data[3]);
 
-        SDL_AtomicLock(&m_FrameQueueLock);
+        if (m_DisplayLayer.status == AVQueuedSampleBufferRenderingStatusFailed) {
+            SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                        "Resetting failed AVSampleBufferDisplay layer");
 
-        int frameDropTarget;
+            // Trigger the main thread to recreate the decoder
-
+            SDL_Event event;
-        // If the queue length history entries are large, be strict
+            event.type = SDL_RENDER_TARGETS_RESET;
-        // about dropping excess frames.
+            SDL_PushEvent(&event);
-        frameDropTarget = 1;
-        for (int i = 0; i < m_FrameQueueHistory.count(); i++) {
-            if (m_FrameQueueHistory[i] <= 1) {
-                // Be lenient as long as the queue length
-                // resolves before the end of frame history
-                frameDropTarget = 3;
-            }
-        }
-
-        if (m_FrameQueueHistory.count() == FRAME_HISTORY_ENTRIES) {
-            m_FrameQueueHistory.dequeue();
-        }
-
-        m_FrameQueueHistory.enqueue(m_FrameQueue.count());
-
-        // Catch up if we're several frames ahead
-        while (m_FrameQueue.count() > frameDropTarget) {
-            AVFrame* frame = m_FrameQueue.dequeue();
-            av_frame_free(&frame);
-        }
-
-        if (m_FrameQueue.isEmpty()) {
-            SDL_AtomicUnlock(&m_FrameQueueLock);
             return;
         }
 
-        // Grab the first frame
-        AVFrame* frame = m_FrameQueue.dequeue();
-        SDL_AtomicUnlock(&m_FrameQueueLock);
-
-        CVPixelBufferRef pixBuf = reinterpret_cast<CVPixelBufferRef>(frame->data[3]);
-
         // If the format has changed or doesn't exist yet, construct it with the
         // pixel buffer data
         if (!m_FormatDesc || !CMVideoFormatDescriptionMatchesImageBuffer(m_FormatDesc, pixBuf)) {
@@ -108,7 +68,6 @@ public:
                 SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                              "CMVideoFormatDescriptionCreateForImageBuffer() failed: %d",
                              status);
-                av_frame_free(&frame);
                 return;
             }
         }
@@ -117,7 +76,7 @@ public:
         CMSampleTimingInfo timingInfo = {
             .duration = kCMTimeInvalid,
             .decodeTimeStamp = kCMTimeInvalid,
-            .presentationTimeStamp = CMTimeMake(vsyncTime, 1000 * 1000 * 1000)
+            .presentationTimeStamp = CMTimeMake(mach_absolute_time(), 1000 * 1000 * 1000)
         };
 
         CMSampleBufferRef sampleBuffer;
@@ -130,45 +89,20 @@ public:
             SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
                          "CMSampleBufferCreateReadyWithImageBuffer() failed: %d",
                          status);
-            av_frame_free(&frame);
             return;
         }
 
         [m_DisplayLayer enqueueSampleBuffer:sampleBuffer];
 
         CFRelease(sampleBuffer);
-        av_frame_free(&frame);
-    }
-
-    static
-    CVReturn
-    displayLinkOutputCallback(
-        CVDisplayLinkRef,
-        const CVTimeStamp* now,
-        const CVTimeStamp* /* vsyncTime */,
-        CVOptionFlags,
-        CVOptionFlags*,
-        void *displayLinkContext)
-    {
-        VTRenderer* me = reinterpret_cast<VTRenderer*>(displayLinkContext);
-
-        // In my testing on macOS 10.13, this callback is invoked about 24 ms
-        // prior to the specified v-sync time (now - vsyncTime). Since this is
-        // greater than the standard v-sync interval (16 ms = 60 FPS), we will
-        // draw using the current host time, rather than the actual v-sync target
-        // time. Because the CVDisplayLink is in sync with the actual v-sync
-        // interval, even if many ms prior, we can safely use the current host time
-        // and get a consistent callback for each v-sync. This reduces video latency
-        // by at least 1 frame vs. rendering with the actual vsyncTime.
-        me->drawFrame(now->hostTime);
-
-        return kCVReturnSuccess;
     }
 
     virtual bool initialize(SDL_Window* window,
                             int videoFormat,
                             int,
-                            int) override
+                            int,
+                            int,
+                            bool) override
     {
         int err;
 
@@ -243,30 +177,23 @@ public:
             return false;
         }
 
-        CVDisplayLinkCreateWithActiveCGDisplays(&m_DisplayLink);
-        CVDisplayLinkSetOutputCallback(m_DisplayLink, displayLinkOutputCallback, this);
-        CVDisplayLinkStart(m_DisplayLink);
-
         return true;
     }
 
     virtual bool prepareDecoderContext(AVCodecContext* context) override
     {
         context->hw_device_ctx = av_buffer_ref(m_HwContext);
+
+        SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION,
+                    "Using VideoToolbox accelerated renderer");
+
         return true;
     }
 
-    virtual void renderFrame(AVFrame* frame) override
+    virtual bool needsTestFrame() override
     {
-        if (m_DisplayLayer.status == AVQueuedSampleBufferRenderingStatusFailed) {
+        // We query VT to determine whether the codec is supported
-            SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+        return false;
-                        "Resetting failed AVSampleBufferDisplay layer");
-            setupDisplayLayer();
-        }
-
-        SDL_AtomicLock(&m_FrameQueueLock);
-        m_FrameQueue.enqueue(frame);
-        SDL_AtomicUnlock(&m_FrameQueueLock);
     }
 
 private:
@@ -292,10 +219,6 @@ private:
     AVSampleBufferDisplayLayer* m_DisplayLayer;
     CMVideoFormatDescriptionRef m_FormatDesc;
     NSView* m_View;
-    CVDisplayLinkRef m_DisplayLink;
-    QQueue<AVFrame*> m_FrameQueue;
-    QQueue<int> m_FrameQueueHistory;
-    SDL_SpinLock m_FrameQueueLock;
 };
 
 IFFmpegRenderer* VTRendererFactory::createRenderer() {

app/streaming/video/ffmpeg.cpp

@@ -13,98 +13,48 @@
 #include "ffmpeg-renderers/vaapi.h"
 #endif
 
-bool FFmpegVideoDecoder::chooseDecoder(
+#ifdef HAVE_LIBVDPAU
-        StreamingPreferences::VideoDecoderSelection vds,
+#include "ffmpeg-renderers/vdpau.h"
-        SDL_Window* window,
-        int videoFormat,
-        int width, int height,
-        AVCodec*& chosenDecoder,
-        const AVCodecHWConfig*& chosenHwConfig,
-        IFFmpegRenderer*& newRenderer)
-{
-    if (videoFormat & VIDEO_FORMAT_MASK_H264) {
-        chosenDecoder = avcodec_find_decoder(AV_CODEC_ID_H264);
-    }
-    else if (videoFormat & VIDEO_FORMAT_MASK_H265) {
-        chosenDecoder = avcodec_find_decoder(AV_CODEC_ID_HEVC);
-    }
-    else {
-        Q_ASSERT(false);
-        chosenDecoder = nullptr;
-    }
-
-    if (!chosenDecoder) {
-        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
-                     "Unable to find decoder for format: %x",
-                     videoFormat);
-        return false;
-    }
-
-    for (int i = 0;; i++) {
-        const AVCodecHWConfig *config = avcodec_get_hw_config(chosenDecoder, i);
-        if (!config || vds == StreamingPreferences::VDS_FORCE_SOFTWARE) {
-            // No matching hardware acceleration support.
-            // This is not an error.
-            chosenHwConfig = nullptr;
-            newRenderer = new SdlRenderer();
-            if (vds != StreamingPreferences::VDS_FORCE_HARDWARE &&
-                    newRenderer->initialize(window, videoFormat, width, height)) {
-                return true;
-            }
-            else {
-                delete newRenderer;
-                newRenderer = nullptr;
-                return false;
-            }
-        }
-
-        if (!(config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX)) {
-            continue;
-        }
-
-        // Look for acceleration types we support
-        switch (config->device_type) {
-#ifdef Q_OS_WIN32
-        case AV_HWDEVICE_TYPE_DXVA2:
-            newRenderer = new DXVA2Renderer();
-            break;
 #endif
-#ifdef Q_OS_DARWIN
-        case AV_HWDEVICE_TYPE_VIDEOTOOLBOX:
-            newRenderer = VTRendererFactory::createRenderer();
-            break;
-#endif
-#ifdef HAVE_LIBVA
-        case AV_HWDEVICE_TYPE_VAAPI:
-            newRenderer = new VAAPIRenderer();
-            break;
-#endif
-        default:
-            continue;
-        }
 
-        if (newRenderer->initialize(window, videoFormat, width, height)) {
+// This is gross but it allows us to use sizeof()
-            chosenHwConfig = config;
+#include "ffmpeg_videosamples.cpp"
-            return true;
+
-        }
+#define FAILED_DECODES_RESET_THRESHOLD 20
-        else {
-            // Failed to initialize
-            delete newRenderer;
-            newRenderer = nullptr;
-        }
-    }
-}
 
 bool FFmpegVideoDecoder::isHardwareAccelerated()
 {
     return m_HwDecodeCfg != nullptr;
 }
 
+enum AVPixelFormat FFmpegVideoDecoder::ffGetFormat(AVCodecContext* context,
+                                                   const enum AVPixelFormat* pixFmts)
+{
+    FFmpegVideoDecoder* decoder = (FFmpegVideoDecoder*)context->opaque;
+    const enum AVPixelFormat *p;
+
+    for (p = pixFmts; *p != -1; p++) {
+        // Only match our hardware decoding codec or SW pixel
+        // format (if not using hardware decoding). It's crucial
+        // to override the default get_format() which will try
+        // to gracefully fall back to software decode and break us.
+        if (*p == (decoder->m_HwDecodeCfg ?
+                   decoder->m_HwDecodeCfg->pix_fmt :
+                   context->pix_fmt)) {
+            return *p;
+        }
+    }
+
+    return AV_PIX_FMT_NONE;
+}
+
 FFmpegVideoDecoder::FFmpegVideoDecoder()
     : m_VideoDecoderCtx(nullptr),
       m_DecodeBuffer(1024 * 1024, 0),
       m_HwDecodeCfg(nullptr),
-      m_Renderer(nullptr)
+      m_Renderer(nullptr),
+      m_ConsecutiveFailedDecodes(0),
+      m_Pacer(nullptr)
 {
     av_init_packet(&m_Pkt);
     SDL_AtomicSet(&m_QueuedFrames, 0);
@@ -114,6 +64,16 @@ FFmpegVideoDecoder::FFmpegVideoDecoder()
 }
 
 FFmpegVideoDecoder::~FFmpegVideoDecoder()
+{
+    reset();
+}
+
+IFFmpegRenderer* FFmpegVideoDecoder::getRenderer()
+{
+    return m_Renderer;
+}
+
+void FFmpegVideoDecoder::reset()
 {
     // Drop any frames still queued to ensure
     // they are properly freed.
@@ -132,29 +92,26 @@ FFmpegVideoDecoder::~FFmpegVideoDecoder()
             dropFrame(&event.user);
         }
         else {
-            SDL_Delay(100);
+            SDL_Delay(10);
             SDL_PumpEvents();
         }
     }
 
-    avcodec_free_context(&m_VideoDecoderCtx);
+    delete m_Pacer;
+    m_Pacer = nullptr;
 
     delete m_Renderer;
+    m_Renderer = nullptr;
+
+    avcodec_free_context(&m_VideoDecoderCtx);
 }
 
-bool FFmpegVideoDecoder::initialize(
+bool FFmpegVideoDecoder::completeInitialization(AVCodec* decoder, SDL_Window* window,
-        StreamingPreferences::VideoDecoderSelection vds,
+                                                int videoFormat, int width, int height,
-        SDL_Window* window,
+                                                int maxFps, bool enableVsync, bool testOnly)
-        int videoFormat,
-        int width,
-        int height,
-        int)
 {
-    AVCodec* decoder;
+    m_Pacer = new Pacer(m_Renderer);
-
+    if (!m_Pacer->initialize(window, maxFps, enableVsync)) {
-    if (!chooseDecoder(vds, window, videoFormat, width, height,
-                       decoder, m_HwDecodeCfg, m_Renderer)) {
-        // Error logged in chooseDecoder()
         return false;
     }
 
@@ -182,12 +139,20 @@ bool FFmpegVideoDecoder::initialize(
     m_VideoDecoderCtx->width = width;
     m_VideoDecoderCtx->height = height;
     m_VideoDecoderCtx->pix_fmt = AV_PIX_FMT_YUV420P; // FIXME: HDR
+    m_VideoDecoderCtx->get_format = ffGetFormat;
 
     // Allow the renderer to attach data to this decoder
     if (!m_Renderer->prepareDecoderContext(m_VideoDecoderCtx)) {
         return false;
     }
 
+    // Nobody must override our ffGetFormat
+    SDL_assert(m_VideoDecoderCtx->get_format == ffGetFormat);
+
+    // Stash a pointer to this object in the context
+    SDL_assert(m_VideoDecoderCtx->opaque == nullptr);
+    m_VideoDecoderCtx->opaque = this;
+
     int err = avcodec_open2(m_VideoDecoderCtx, decoder, nullptr);
     if (err < 0) {
         SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
@@ -196,9 +161,155 @@ bool FFmpegVideoDecoder::initialize(
         return false;
     }
 
+    // FFMpeg doesn't completely initialize the codec until the codec
+    // config data comes in. This would be too late for us to change
+    // our minds on the selected video codec, so we'll do a trial run
+    // now to see if things will actually work when the video stream
+    // comes in.
+    if (testOnly) {
+        if (videoFormat & VIDEO_FORMAT_MASK_H264) {
+            m_Pkt.data = (uint8_t*)k_H264TestFrame;
+            m_Pkt.size = sizeof(k_H264TestFrame);
+        }
+        else {
+            m_Pkt.data = (uint8_t*)k_HEVCTestFrame;
+            m_Pkt.size = sizeof(k_HEVCTestFrame);
+        }
+
+        err = avcodec_send_packet(m_VideoDecoderCtx, &m_Pkt);
+        if (err < 0) {
+            char errorstring[512];
+            av_strerror(err, errorstring, sizeof(errorstring));
+            SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                        "Test decode failed: %s", errorstring);
+            return false;
+        }
+    }
+
+#ifdef QT_DEBUG
+    // Restore default log level before streaming
+    av_log_set_level(AV_LOG_INFO);
+#endif
+
     return true;
 }
 
+IFFmpegRenderer* FFmpegVideoDecoder::createAcceleratedRenderer(const AVCodecHWConfig* hwDecodeCfg)
+{
+    if (!(hwDecodeCfg->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX)) {
+        return nullptr;
+    }
+
+    // Look for acceleration types we support
+    switch (hwDecodeCfg->device_type) {
+#ifdef Q_OS_WIN32
+    case AV_HWDEVICE_TYPE_DXVA2:
+        return new DXVA2Renderer();
+#endif
+#ifdef Q_OS_DARWIN
+    case AV_HWDEVICE_TYPE_VIDEOTOOLBOX:
+        return VTRendererFactory::createRenderer();
+#endif
+#ifdef HAVE_LIBVA
+    case AV_HWDEVICE_TYPE_VAAPI:
+        return new VAAPIRenderer();
+#endif
+#ifdef HAVE_LIBVDPAU
+    case AV_HWDEVICE_TYPE_VDPAU:
+        return new VDPAURenderer();
+#endif
+    default:
+        return nullptr;
+    }
+}
+
+bool FFmpegVideoDecoder::initialize(
+        StreamingPreferences::VideoDecoderSelection vds,
+        SDL_Window* window,
+        int videoFormat,
+        int width,
+        int height,
+        int maxFps,
+        bool enableVsync)
+{
+    AVCodec* decoder;
+
+#ifdef QT_DEBUG
+    // Increase log level during initialization
+    av_log_set_level(AV_LOG_DEBUG);
+#endif
+
+    if (videoFormat & VIDEO_FORMAT_MASK_H264) {
+        decoder = avcodec_find_decoder(AV_CODEC_ID_H264);
+    }
+    else if (videoFormat & VIDEO_FORMAT_MASK_H265) {
+        decoder = avcodec_find_decoder(AV_CODEC_ID_HEVC);
+    }
+    else {
+        Q_ASSERT(false);
+        decoder = nullptr;
+    }
+
+    if (!decoder) {
+        SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                     "Unable to find decoder for format: %x",
+                     videoFormat);
+        return false;
+    }
+
+    for (int i = 0;; i++) {
+        const AVCodecHWConfig *config = avcodec_get_hw_config(decoder, i);
+        if (!config || vds == StreamingPreferences::VDS_FORCE_SOFTWARE) {
+            // No matching hardware acceleration support.
+            // This is not an error.
+            m_HwDecodeCfg = nullptr;
+            m_Renderer = new SdlRenderer();
+            if (vds != StreamingPreferences::VDS_FORCE_HARDWARE &&
+                    m_Renderer->initialize(window, videoFormat, width, height, maxFps, enableVsync) &&
+                    completeInitialization(decoder, window, videoFormat, width, height, maxFps, enableVsync, false)) {
+                return true;
+            }
+            else {
+                reset();
+                return false;
+            }
+        }
+
+        m_Renderer = createAcceleratedRenderer(config);
+        if (!m_Renderer) {
+            continue;
+        }
+
+        m_HwDecodeCfg = config;
+        // Initialize the hardware codec and submit a test frame if the renderer needs it
+        if (m_Renderer->initialize(window, videoFormat, width, height, maxFps, enableVsync) &&
+                completeInitialization(decoder, window, videoFormat, width, height, maxFps, enableVsync, m_Renderer->needsTestFrame())) {
+            if (m_Renderer->needsTestFrame()) {
+                // The test worked, so now let's initialize it for real
+                reset();
+                if ((m_Renderer = createAcceleratedRenderer(config)) != nullptr &&
+                        m_Renderer->initialize(window, videoFormat, width, height, maxFps, enableVsync) &&
+                        completeInitialization(decoder, window, videoFormat, width, height, maxFps, enableVsync, false)) {
+                    return true;
+                }
+                else {
+                    SDL_LogCritical(SDL_LOG_CATEGORY_APPLICATION,
+                                    "Decoder failed to initialize after successful test");
+                    reset();
+                }
+            }
+            else {
+                // No test required. Good to go now.
+                return true;
+            }
+        }
+        else {
+            // Failed to initialize or test frame failed, so keep looking
+            reset();
+        }
+    }
+}
+
 int FFmpegVideoDecoder::submitDecodeUnit(PDECODE_UNIT du)
 {
     PLENTRY entry = du->bufferList;
@@ -224,12 +335,23 @@ int FFmpegVideoDecoder::submitDecodeUnit(PDECODE_UNIT du)
 
     err = avcodec_send_packet(m_VideoDecoderCtx, &m_Pkt);
     if (err < 0) {
-        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
-                    "Decoding failed: %d", err);
         char errorstring[512];
         av_strerror(err, errorstring, sizeof(errorstring));
         SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
-                    "Decoding failed: %s", errorstring);
+                    "avcodec_send_packet() failed: %s", errorstring);
+
+        // If we've failed a bunch of decodes in a row, the decoder/renderer is
+        // clearly unhealthy, so let's generate a synthetic reset event to trigger
+        // the event loop to destroy and recreate the decoder.
+        if (++m_ConsecutiveFailedDecodes == FAILED_DECODES_RESET_THRESHOLD) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "Resetting decoder due to consistent failure");
+
+            SDL_Event event;
+            event.type = SDL_RENDER_DEVICE_RESET;
+            SDL_PushEvent(&event);
+        }
+
         return DR_NEED_IDR;
     }
 
@@ -244,12 +366,29 @@ int FFmpegVideoDecoder::submitDecodeUnit(PDECODE_UNIT du)
 
     err = avcodec_receive_frame(m_VideoDecoderCtx, frame);
     if (err == 0) {
+        // Reset failed decodes count if we reached this far
+        m_ConsecutiveFailedDecodes = 0;
+
         // Queue the frame for rendering from the main thread
         SDL_AtomicIncRef(&m_QueuedFrames);
         queueFrame(frame);
     }
     else {
         av_frame_free(&frame);
+
+        char errorstring[512];
+        av_strerror(err, errorstring, sizeof(errorstring));
+        SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION,
+                    "avcodec_receive_frame() failed: %s", errorstring);
+
+        if (++m_ConsecutiveFailedDecodes == FAILED_DECODES_RESET_THRESHOLD) {
+            SDL_LogError(SDL_LOG_CATEGORY_APPLICATION,
+                         "Resetting decoder due to consistent failure");
+
+            SDL_Event event;
+            event.type = SDL_RENDER_DEVICE_RESET;
+            SDL_PushEvent(&event);
+        }
     }
 
     return DR_OK;
@@ -259,7 +398,7 @@ int FFmpegVideoDecoder::submitDecodeUnit(PDECODE_UNIT du)
 void FFmpegVideoDecoder::renderFrame(SDL_UserEvent* event)
 {
     AVFrame* frame = reinterpret_cast<AVFrame*>(event->data1);
-    m_Renderer->renderFrame(frame);
+    m_Pacer->submitFrame(frame);
     SDL_AtomicDecRef(&m_QueuedFrames);
 }
 
@@ -267,6 +406,15 @@ void FFmpegVideoDecoder::renderFrame(SDL_UserEvent* event)
 void FFmpegVideoDecoder::dropFrame(SDL_UserEvent* event)
 {
     AVFrame* frame = reinterpret_cast<AVFrame*>(event->data1);
+    // If Pacer is using a frame queue, we can just queue the
+    // frame and let it decide whether to drop or not. If Pacer
+    // is submitting directly for rendering, we drop the frame
+    // ourselves.
+    if (m_Pacer->isUsingFrameQueue()) {
+        m_Pacer->submitFrame(frame);
+    }
+    else {
         av_frame_free(&frame);
+    }
     SDL_AtomicDecRef(&m_QueuedFrames);
 }

app/streaming/video/ffmpeg.h

@@ -2,6 +2,7 @@
 
 #include "decoder.h"
 #include "ffmpeg-renderers/renderer.h"
+#include "ffmpeg-renderers/pacer/pacer.h"
 
 extern "C" {
 #include <libavcodec/avcodec.h>
@@ -16,22 +17,27 @@ public:
                             int videoFormat,
                             int width,
                             int height,
-                            int frameRate) override;
+                            int maxFps,
+                            bool enableVsync) override;
     virtual bool isHardwareAccelerated() override;
     virtual int submitDecodeUnit(PDECODE_UNIT du) override;
     virtual void renderFrame(SDL_UserEvent* event) override;
     virtual void dropFrame(SDL_UserEvent* event) override;
 
+    virtual IFFmpegRenderer* getRenderer();
+
 private:
-    bool
+    bool completeInitialization(AVCodec* decoder, SDL_Window* window,
-    chooseDecoder(
+                                int videoFormat, int width, int height,
-        StreamingPreferences::VideoDecoderSelection vds,
+                                int maxFps, bool enableVsync, bool testOnly);
-        SDL_Window* window,
+
-        int videoFormat,
+    IFFmpegRenderer* createAcceleratedRenderer(const AVCodecHWConfig* hwDecodeCfg);
-        int width, int height,
+
-        AVCodec*& chosenDecoder,
+    void reset();
-        const AVCodecHWConfig*& chosenHwConfig,
+
-        IFFmpegRenderer*& newRenderer);
+    static
+    enum AVPixelFormat ffGetFormat(AVCodecContext* context,
+                                   const enum AVPixelFormat* pixFmts);
 
     AVPacket m_Pkt;
     AVCodecContext* m_VideoDecoderCtx;
@@ -39,4 +45,9 @@ private:
     const AVCodecHWConfig* m_HwDecodeCfg;
     IFFmpegRenderer* m_Renderer;
     SDL_atomic_t m_QueuedFrames;
+    int m_ConsecutiveFailedDecodes;
+    Pacer* m_Pacer;
+
+    static const uint8_t k_H264TestFrame[];
+    static const uint8_t k_HEVCTestFrame[];
 };

app/streaming/video/ffmpeg_videosamples.cpp

@@ -0,0 +1,19 @@
+#include "ffmpeg.h"
+
+// 720p 60 FPS H.264 with 1 reference frame
+const uint8_t FFmpegVideoDecoder::k_H264TestFrame[] = {
+    0x00, 0x00, 0x00, 0x01, 0x67, 0x64, 0x00, 0x20, 0xac, 0x2b, 0x40, 0x28, 0x02, 0xdd, 0x80, 0xb5, 0x06, 0x06, 0x06, 0xa5, 0x00, 0x00, 0x03, 0x03, 0xe8, 0x00, 0x01, 0xd4, 0xc0, 0x8f, 0x4a, 0xa0,
+    0x00, 0x00, 0x00, 0x01, 0x68, 0xee, 0x3c, 0xb0,
+    0x00, 0x00, 0x00, 0x01, 0x65, 0xb8, 0x02, 0x01, 0x67, 0x25, 0x1b, 0xf4, 0xfa, 0x7d, 0x40, 0x1a, 0x78, 0xe5, 0x10, 0x52, 0xc2, 0xee, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0xc6, 0xef, 0xbb, 0x81, 0x85, 0x2d, 0x47, 0xda, 0xca, 0x4c, 0x00, 0x00, 0x03, 0x00, 0x02, 0x7b, 0xcf, 0x80, 0x00, 0x45, 0x40, 0x01, 0x8d, 0xa6, 0x00, 0x01, 0x64, 0x00, 0x0e, 0x03, 0xc8, 0x00, 0x0e, 0x10, 0x00, 0xbd, 0xc5, 0x00, 0x01, 0x11, 0x00, 0x0e, 0xa3, 0x80, 0x00, 0x38, 0xa0,
+    0x00, 0x00, 0x01, 0x65, 0x00, 0x6e, 0x2e, 0x00, 0x83, 0x7f, 0xb4, 0x8e, 0x79, 0xa5, 0xff, 0x84, 0x3f, 0x7f, 0x34, 0x3f, 0x97, 0x1b, 0xaf, 0x31, 0x5f, 0x6e, 0xaa, 0xb6, 0xac, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x78, 0x36, 0x9d, 0xa4, 0x65, 0xf9, 0x1e, 0x5b, 0x3a, 0xb0, 0x40, 0x00, 0x00, 0x03, 0x00, 0x00, 0x41, 0x80, 0x00, 0xc5, 0xc8, 0x00, 0x00, 0xfa, 0x00, 0x03, 0x24, 0x00, 0x0e, 0x20, 0x00, 0x3f, 0x80, 0x01, 0x32, 0x00, 0x08, 0x68, 0x00, 0x3e, 0xc0, 0x03, 0x8a,
+    0x00, 0x00, 0x01, 0x65, 0x00, 0x37, 0x0b, 0x80, 0x21, 0x7f, 0xe3, 0x85, 0x1c, 0xd9, 0xff, 0xe1, 0x1b, 0x01, 0xfa, 0xc0, 0x3e, 0x11, 0x7e, 0xfe, 0x45, 0x5c, 0x43, 0x69, 0x31, 0x4b, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x02, 0x24, 0xae, 0x1a, 0xbb, 0xae, 0x75, 0x9e, 0x35, 0xae, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x03, 0x64, 0x00, 0x09, 0x98, 0x00, 0x1e, 0xc0, 0x00, 0x64, 0x80, 0x01, 0xc4, 0x00, 0x07, 0xf0, 0x00, 0x42, 0xf0, 0x00, 0x00, 0xe1, 0x98, 0x00, 0x05, 0x4b, 0x28, 0x00, 0x06, 0x04,
+    0x00, 0x00, 0x01, 0x65, 0x00, 0x14, 0xa2, 0xe0, 0x08, 0x5f, 0xe3, 0x85, 0x1c, 0xd9, 0xff, 0xe1, 0x1b, 0x01, 0xfa, 0xc0, 0x3e, 0x11, 0x7e, 0xfe, 0x45, 0x5c, 0x43, 0x69, 0x31, 0x4b, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x02, 0x24, 0xb9, 0x7d, 0xb4, 0x70, 0x4d, 0x15, 0xc5, 0x0a, 0x4e, 0x60, 0x00, 0x00, 0x03, 0x00, 0x00, 0x26, 0xa8, 0xb0, 0x00, 0x13, 0xcd, 0xcc, 0x00, 0x07, 0x48, 0x88, 0x00, 0x06, 0x29, 0x69, 0x00, 0x01, 0x16, 0xac, 0x80, 0x00, 0x9e, 0x4e, 0x80, 0x00, 0x3f, 0x84, 0x20, 0x00, 0x6f, 0x41, 0xa0, 0x00, 0x20, 0x00, 0x02, 0x16, 0xb8, 0x00, 0x08, 0x08
+};
+
+// 720p 60 FPS HEVC with 1 reference frame
+const uint8_t FFmpegVideoDecoder::k_HEVCTestFrame[] = {
+    0x00, 0x00, 0x00, 0x01, 0x40, 0x01, 0x0c, 0x01, 0xff, 0xff, 0x01, 0x40, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x78, 0xac, 0x09,
+    0x00, 0x00, 0x00, 0x01, 0x42, 0x01, 0x01, 0x01, 0x40, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x78, 0xa0, 0x02, 0x80, 0x80, 0x2e, 0x1f, 0x13, 0x96, 0xb4, 0xa4, 0x25, 0x92, 0xe3, 0x01, 0x6a, 0x0c, 0x0c, 0x0d, 0x48, 0x20, 0x00, 0x00, 0x03, 0x00, 0x20, 0x00, 0x00, 0x07, 0x85, 0xf1, 0xa2, 0xd0,
+    0x00, 0x00, 0x00, 0x01, 0x44, 0x01, 0xc0, 0xf7, 0xc0, 0xcc, 0x90,
+    0x00, 0x00, 0x00, 0x01, 0x2a, 0x01, 0xac, 0x0c, 0xf0, 0xe6, 0xf6, 0x5e, 0xff, 0xd9, 0x97, 0xeb, 0x44, 0x7d, 0x1f, 0xa4, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x03, 0x52, 0x7c, 0x4e, 0x5b, 0xfe, 0xbe, 0xc2, 0x87, 0x20, 0x00, 0x00, 0x03, 0x00, 0x00, 0x03, 0x00, 0x2b, 0x60,
+};

app/streaming/video/sl.cpp

@@ -28,7 +28,7 @@ SLVideoDecoder::isHardwareAccelerated()
 bool
 SLVideoDecoder::initialize(StreamingPreferences::VideoDecoderSelection vds,
                            SDL_Window*,
-                           int videoFormat, int, int, int frameRate)
+                           int videoFormat, int, int, int frameRate, bool)
 {
     // SLVideo only supports hardware decoding
     if (vds == StreamingPreferences::VDS_FORCE_SOFTWARE) {

app/streaming/video/sl.h

@@ -14,7 +14,8 @@ public:
                             int videoFormat,
                             int width,
                             int height,
-                            int frameRate);
+                            int frameRate,
+                            bool enableVsync);
     virtual bool isHardwareAccelerated();
     virtual int submitDecodeUnit(PDECODE_UNIT du);
 

libs/mac/Frameworks/SDL2.framework/Headers

@@ -0,0 +1 @@
+Versions/Current/Headers

libs/mac/Frameworks/SDL2.framework/Resources

@@ -0,0 +1 @@
+Versions/Current/Resources

libs/mac/Frameworks/SDL2.framework/SDL2

@@ -0,0 +1 @@
+Versions/Current/SDL2