Switch to a pull-based renderer and render from CADisplayLink callback

Fixes #464

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,48 @@
+---
+name: Bug report
+about: Follow the troubleshooting guide before reporting a bug
+
+---
+**READ ME FIRST!**
+If you're here because something basic is not working (like gamepad input, video, or similar), it's probably something specific to your setup, so make sure you've gone through the Troubleshooting Guide first: https://github.com/moonlight-stream/moonlight-docs/wiki/Troubleshooting
+
+If you still have trouble with basic functionality after following the guide, join our Discord server where there are many other volunteers who can help (or direct you back here if it looks like a Moonlight bug after all). https://moonlight-stream.org/discord
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**Steps to reproduce**
+Any special steps that are required for the bug to appear.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem. If the issue is related to video glitching or poor quality, please include screenshots.
+
+**Affected games**
+List the games you've tried that exhibit the issue. To see if the issue is game-specific, try streaming Steam Big Picture with Moonlight and see if the issue persists there.
+
+**Other Moonlight clients**
+- Does the issue occur when using Moonlight on PC or Android?
+
+**Moonlight settings (please complete the following information)**
+- Have any settings been adjusted from defaults?
+- If so, which settings have been changed?
+- Does the problem still occur after reverting settings back to default?
+
+**Gamepad-related issues (please complete if problem is gamepad-related)**
+- Do you have any gamepads connected to your host PC directly?
+- If gamepad input is not working, does it work if you use Moonlight's on-screen controls?
+- Does the problem still remain if you stream the desktop and use https://html5gamepad.com to test your gamepad?
+  - Instructions for streaming the desktop can be found here: https://github.com/moonlight-stream/moonlight-docs/wiki/Setup-Guide
+
+**Device details (please complete the following information)**
+ - iOS/tvOS version: [e.g. iOS 14.2]
+ - Device model: [e.g. iPhone 11 Pro]
+
+**Server PC details (please complete the following information)**
+ - OS: [e.g. Windows 10 1809]
+ - GeForce Experience version: [e.g. 3.16.0.140]
+ - Nvidia GPU driver: [e.g. 417.35]
+ - Antivirus and firewall software: [e.g. Windows Defender and Windows Firewall]
+
+**Additional context**
+Anything else you think may be relevant to the issue or special about your specific setup.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: false

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.github/auto-comment.yml

@@ -0,0 +1,4 @@
+issuesOpened: >
+  If this is a question about Moonlight or you need help troubleshooting a streaming problem, please use the help channels on our [Discord server](https://moonlight-stream.org/discord) instead of GitHub issues. There are many more people available on Discord to help you and answer your questions.<br /><br />
+  This issue tracker should only be used for specific bugs or feature requests.<br /><br />
+  Thank you, and happy streaming!

.github/lock.yml

@@ -0,0 +1,25 @@
+# Configuration for Lock Threads - https://github.com/dessant/lock-threads-app
+
+# Number of days of inactivity before a closed issue or pull request is locked
+daysUntilLock: 60
+
+# Skip issues and pull requests created before a given timestamp. Timestamp must
+# follow ISO 8601 (`YYYY-MM-DD`). Set to `false` to disable
+skipCreatedBefore: false
+
+# Issues and pull requests with these labels will be ignored. Set to `[]` to disable
+exemptLabels:
+  - enhancement
+
+# Label to add before locking, such as `outdated`. Set to `false` to disable
+lockLabel: false
+
+# Comment to post before locking. Set to `false` to disable
+lockComment: >
+  This issue has been automatically locked since there has not been
+  any recent activity after it was closed. If you experience an issue
+  that looks similar to this one, please open a new issue with details
+  about your specific streaming setup.
+
+# Don't lock PRs
+only: issues

.gitmodules

@@ -1,3 +1,6 @@
 [submodule "moonlight-common/moonlight-common-c"]
 	path = moonlight-common/moonlight-common-c
 	url = https://github.com/moonlight-stream/moonlight-common-c.git
+[submodule "X1Kit"]
+	path = X1Kit
+	url = https://github.com/cgutman/X1Kit.git

Launch Screen.xib

@@ -1,51 +1,19 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
-<document type="com.apple.InterfaceBuilder3.CocoaTouch.XIB" version="3.0" toolsVersion="8191" systemVersion="15A279b" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" launchScreen="YES" useTraitCollections="YES">
+<?xml version="1.0" encoding="UTF-8"?>
+<document type="com.apple.InterfaceBuilder3.CocoaTouch.XIB" version="3.0" toolsVersion="15505" targetRuntime="iOS.CocoaTouch" propertyAccessControl="none" useAutolayout="YES" launchScreen="YES" useTraitCollections="YES" colorMatched="YES">
+    <device id="retina6_1" orientation="landscape" appearance="light"/>
     <dependencies>
         <deployment identifier="iOS"/>
-        <plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="8154"/>
+        <plugIn identifier="com.apple.InterfaceBuilder.IBCocoaTouchPlugin" version="15509"/>
+        <capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
     </dependencies>
     <objects>
         <placeholder placeholderIdentifier="IBFilesOwner" id="-1" userLabel="File's Owner"/>
         <placeholder placeholderIdentifier="IBFirstResponder" id="-2" customClass="UIResponder"/>
-        <navigationController id="vV9-32-ww6">
-            <simulatedOrientationMetrics key="simulatedOrientationMetrics" orientation="landscapeRight"/>
-            <navigationBar key="navigationBar" contentMode="scaleToFill" id="y2P-aE-Haa">
-                <rect key="frame" x="0.0" y="0.0" width="320" height="44"/>
-                <autoresizingMask key="autoresizingMask"/>
-                <animations/>
-                <color key="barTintColor" white="0.66666666666666663" alpha="1" colorSpace="calibratedWhite"/>
-            </navigationBar>
-            <viewControllers>
-                <viewController id="l6V-ue-5Om">
-                    <layoutGuides>
-                        <viewControllerLayoutGuide type="top" id="rEZ-vx-akr"/>
-                        <viewControllerLayoutGuide type="bottom" id="gBD-cy-cth"/>
-                    </layoutGuides>
-                    <view key="view" contentMode="scaleToFill" id="6lF-fc-kpK">
-                        <autoresizingMask key="autoresizingMask" widthSizable="YES" heightSizable="YES"/>
-                        <animations/>
-                        <color key="backgroundColor" white="0.33333333333333331" alpha="1" colorSpace="calibratedWhite"/>
-                    </view>
-                    <navigationItem key="navigationItem" id="rW0-BH-s7l">
-                        <barButtonItem key="leftBarButtonItem" enabled="NO" id="d9R-Zo-Uml">
-                            <button key="customView" opaque="NO" contentMode="center" contentHorizontalAlignment="center" contentVerticalAlignment="center" adjustsImageWhenDisabled="NO" lineBreakMode="middleTruncation" id="9JC-B5-BAE">
-                                <rect key="frame" x="20" y="-1" width="109" height="46"/>
-                                <autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"/>
-                                <animations/>
-                                <fontDescription key="fontDescription" type="system" pointSize="16"/>
-                                <state key="normal" title="Moonlight" image="Logo">
-                                    <color key="titleColor" white="0.66666666666666663" alpha="1" colorSpace="calibratedWhite"/>
-                                    <color key="titleShadowColor" white="0.66666666666666663" alpha="1" colorSpace="calibratedWhite"/>
-                                </state>
-                                <variation key="widthClass=regular" preservesSuperviewLayoutMargins="NO"/>
-                            </button>
-                        </barButtonItem>
-                    </navigationItem>
-                </viewController>
-            </viewControllers>
-        </navigationController>
+        <view contentMode="scaleToFill" id="VWy-YH-qYj">
+            <rect key="frame" x="0.0" y="0.0" width="896" height="414"/>
+            <autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMaxY="YES"/>
+            <color key="backgroundColor" white="0.33333333329999998" alpha="1" colorSpace="custom" customColorSpace="genericGamma22GrayColorSpace"/>
+            <point key="canvasLocation" x="33" y="187"/>
+        </view>
     </objects>
-    <resources>
-        <image name="Logo" width="32" height="32"/>
-    </resources>
 </document>

Limelight/AppDelegate.h

@@ -6,19 +6,13 @@
 //  Copyright (c) 2014 Moonlight Stream. All rights reserved.
 //
 
-#if TARGET_OS_IPHONE
 #import <UIKit/UIKit.h>
 
 @interface AppDelegate : UIResponder <UIApplicationDelegate>
 
 @property (strong, nonatomic) UIWindow *window;
-#else
-#import <Cocoa/Cocoa.h>
-@interface AppDelegate : NSObject <NSApplicationDelegate>
-
-@property (readonly, strong) NSPersistentContainer *persistentContainer;
-@property (strong, nonatomic) NSWindow *window;
-#endif
+@property (strong, nonatomic) NSString *pcUuidToLoad;
+@property (strong, nonatomic) void (^shortcutCompletionHandler)(BOOL);
 
 @property (readonly, strong, nonatomic) NSManagedObjectContext *managedObjectContext;
 @property (readonly, strong, nonatomic) NSManagedObjectModel *managedObjectModel;

Limelight/AppDelegate.m

@@ -18,19 +18,27 @@ static NSOperationQueue* mainQueue;
 
 #if TARGET_OS_TV
 static NSString* DB_NAME = @"Moonlight_tvOS.bin";
-#elif TARGET_OS_IPHONE
-static NSString* DB_NAME = @"Limelight_iOS.sqlite";
 #else
-static NSString* DB_NAME = @"moonlight_mac.sqlite";
+static NSString* DB_NAME = @"Limelight_iOS.sqlite";
 #endif
 
-#if TARGET_OS_IPHONE
-- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
-{
-    
+- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
+#if !TARGET_OS_TV
+    UIApplicationShortcutItem* shortcut = [launchOptions valueForKey:UIApplicationLaunchOptionsShortcutItemKey];
+    if (shortcut != nil) {
+        _pcUuidToLoad = (NSString*)[shortcut.userInfo objectForKey:@"UUID"];
+    }
+#endif
     return YES;
 }
 
+#if !TARGET_OS_TV
+- (void)application:(UIApplication *)application performActionForShortcutItem:(UIApplicationShortcutItem *)shortcutItem completionHandler:(void (^)(BOOL succeeded))completionHandler {
+    _pcUuidToLoad = (NSString*)[shortcutItem.userInfo objectForKey:@"UUID"];
+    _shortcutCompletionHandler = completionHandler;
+}
+#endif
+
 - (void)applicationWillResignActive:(UIApplication *)application
 {
     // Sent when the application is about to move from active to inactive state. This can occur for certain types of temporary interruptions (such as an incoming phone call or SMS message) or when the user quits the application and it begins the transition to the background state.
@@ -58,17 +66,6 @@ static NSString* DB_NAME = @"moonlight_mac.sqlite";
     // Saves changes in the application's managed object context before the application terminates.
     [self saveContext];
 }
-#else
-- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
-    // Insert code here to initialize your application
-}
-
-- (void)applicationWillTerminate:(NSNotification *)aNotification {
-    // Insert code here to tear down your application
-    [self saveContext];
-}
-
-#endif
 
 - (void)saveContext
 {

Limelight/Crypto/CryptoManager.m

@@ -9,7 +9,6 @@
 #import "CryptoManager.h"
 #import "mkcert.h"
 
-#include <openssl/aes.h>
 #include <openssl/sha.h>
 #include <openssl/x509.h>
 #include <openssl/pem.h>
@@ -45,46 +44,47 @@ static NSData* p12 = nil;
 }
 
 - (NSData*) aesEncrypt:(NSData*)data withKey:(NSData*)key {
-    AES_KEY aesKey;
-    AES_set_encrypt_key([key bytes], 128, &aesKey);
-    int size = [self getEncryptSize:data];
-    unsigned char* buffer = malloc(size);
-    unsigned char* blockRoundedBuffer = calloc(1, size);
-    memcpy(blockRoundedBuffer, [data bytes], [data length]);
+    EVP_CIPHER_CTX* cipher;
+    int ciphertextLen;
+
+    cipher = EVP_CIPHER_CTX_new();
+
+    EVP_EncryptInit(cipher, EVP_aes_128_ecb(), [key bytes], NULL);
+    EVP_CIPHER_CTX_set_padding(cipher, 0);
+
+    NSMutableData* ciphertext = [NSMutableData dataWithLength:[data length]];
+    EVP_EncryptUpdate(cipher,
+                      [ciphertext mutableBytes],
+                      &ciphertextLen,
+                      [data bytes],
+                      (int)[data length]);
+    assert(ciphertextLen == [ciphertext length]);
+
+    EVP_CIPHER_CTX_free(cipher);
     
-    // AES_encrypt only encrypts the first 16 bytes so iterate the entire buffer
-    int blockOffset = 0;
-    while (blockOffset < size) {
-        AES_encrypt(blockRoundedBuffer + blockOffset, buffer + blockOffset, &aesKey);
-        blockOffset += 16;
-    }
-    
-    NSData* encryptedData = [NSData dataWithBytes:buffer length:size];
-    free(buffer);
-    free(blockRoundedBuffer);
-    return encryptedData;
+    return ciphertext;
 }
 
 - (NSData*) aesDecrypt:(NSData*)data withKey:(NSData*)key {
-    AES_KEY aesKey;
-    AES_set_decrypt_key([key bytes], 128, &aesKey);
-    unsigned char* buffer = malloc([data length]);
-    
-    // AES_decrypt only decrypts the first 16 bytes so iterate the entire buffer
-    int blockOffset = 0;
-    while (blockOffset < [data length]) {
-        AES_decrypt([data bytes] + blockOffset, buffer + blockOffset, &aesKey);
-        blockOffset += 16;
-    }
-    
-    NSData* decryptedData = [NSData dataWithBytes:buffer length:[data length]];
-    free(buffer);
-    return decryptedData;
-}
+    EVP_CIPHER_CTX* cipher;
+    int plaintextLen;
 
-- (int) getEncryptSize:(NSData*)data {
-    // the size is the length of the data ceiling to the nearest 16 bytes
-    return (((int)[data length] + 15) / 16) * 16;
+    cipher = EVP_CIPHER_CTX_new();
+
+    EVP_DecryptInit(cipher, EVP_aes_128_ecb(), [key bytes], NULL);
+    EVP_CIPHER_CTX_set_padding(cipher, 0);
+
+    NSMutableData* plaintext = [NSMutableData dataWithLength:[data length]];
+    EVP_DecryptUpdate(cipher,
+                      [plaintext mutableBytes],
+                      &plaintextLen,
+                      [data bytes],
+                      (int)[data length]);
+    assert(plaintextLen == [plaintext length]);
+
+    EVP_CIPHER_CTX_free(cipher);
+    
+    return plaintext;
 }
 
 + (NSData*) pemToDer:(NSData*)pemCertBytes {
@@ -227,13 +227,28 @@ static NSData* p12 = nil;
         Log(LOG_E, @"Unable to parse certificate in memory!");
         return NULL;
     }
-    return [NSData dataWithBytes:x509->signature->data length:x509->signature->length];
+    
+#if (OPENSSL_VERSION_NUMBER < 0x10002000L)
+    ASN1_BIT_STRING *asnSignature = x509->signature;
+#elif (OPENSSL_VERSION_NUMBER < 0x10100000L)
+    ASN1_BIT_STRING *asnSignature;
+    X509_get0_signature(&asnSignature, NULL, x509);
+#else
+    const ASN1_BIT_STRING *asnSignature;
+    X509_get0_signature(&asnSignature, NULL, x509);
+#endif
+    
+    NSData* sig = [NSData dataWithBytes:asnSignature->data length:asnSignature->length];
+    
+    X509_free(x509);
+    
+    return sig;
 }
 
 + (NSData*)getKeyFromCertKeyPair:(CertKeyPair*)certKeyPair {
     BIO* bio = BIO_new(BIO_s_mem());
     
-    PEM_write_bio_PrivateKey(bio, certKeyPair->pkey, NULL, NULL, 0, NULL, NULL);
+    PEM_write_bio_PrivateKey_traditional(bio, certKeyPair->pkey, NULL, NULL, 0, NULL, NULL);
     
     BUF_MEM* mem;
     BIO_get_mem_ptr(bio, &mem);

Limelight/Crypto/mkcert.c

@@ -5,19 +5,60 @@
 #include <stdlib.h>
 
 #include <openssl/pem.h>
-#include <openssl/conf.h>
-#include <openssl/pkcs12.h>
-
-#ifndef OPENSSL_NO_ENGINE
-#include <openssl/engine.h>
-#endif
+#include <openssl/rsa.h>
+#include <openssl/x509.h>
+#include <openssl/rand.h>
 
 static const int NUM_BITS = 2048;
 static const int SERIAL = 0;
-static const int NUM_YEARS = 10;
+static const int NUM_YEARS = 20;
 
-int mkcert(X509 **x509p, EVP_PKEY **pkeyp, int bits, int serial, int years);
-int add_ext(X509 *cert, int nid, char *value);
+void mkcert(X509 **x509p, EVP_PKEY **pkeyp, int bits, int serial, int years) {
+    X509* cert = X509_new();
+    
+    EVP_PKEY_CTX* ctx = EVP_PKEY_CTX_new_id(EVP_PKEY_RSA, NULL);
+
+    EVP_PKEY_keygen_init(ctx);
+    EVP_PKEY_CTX_set_rsa_keygen_bits(ctx, bits);
+
+    // pk must be initialized on input
+    EVP_PKEY* pk = NULL;
+    EVP_PKEY_keygen(ctx, &pk);
+
+    EVP_PKEY_CTX_free(ctx);
+    
+    X509_set_version(cert, 2);
+    ASN1_INTEGER_set(X509_get_serialNumber(cert), serial);
+#if OPENSSL_VERSION_NUMBER < 0x10100000L
+    X509_gmtime_adj(X509_get_notBefore(cert), 0);
+    X509_gmtime_adj(X509_get_notAfter(cert), 60 * 60 * 24 * 365 * years);
+#else
+    ASN1_TIME* before = ASN1_STRING_dup(X509_get0_notBefore(cert));
+    ASN1_TIME* after = ASN1_STRING_dup(X509_get0_notAfter(cert));
+
+    X509_gmtime_adj(before, 0);
+    X509_gmtime_adj(after, 60 * 60 * 24 * 365 * years);
+
+    X509_set1_notBefore(cert, before);
+    X509_set1_notAfter(cert, after);
+
+    ASN1_STRING_free(before);
+    ASN1_STRING_free(after);
+#endif
+
+    X509_set_pubkey(cert, pk);
+
+    X509_NAME* name = X509_get_subject_name(cert);
+    X509_NAME_add_entry_by_txt(name, "CN", MBSTRING_ASC,
+                               (const unsigned char*)"NVIDIA GameStream Client",
+                               -1, -1, 0);
+    X509_set_issuer_name(cert, name);
+
+    X509_sign(cert, pk, EVP_sha256());
+    
+    *x509p = cert;
+    *pkeyp = pk;
+}
 
 struct CertKeyPair generateCertKeyPair(void) {
     BIO *bio_err;
@@ -25,32 +66,15 @@ struct CertKeyPair generateCertKeyPair(void) {
     EVP_PKEY *pkey = NULL;
     PKCS12 *p12 = NULL;
    
-    CRYPTO_mem_ctrl(CRYPTO_MEM_CHECK_ON);
     bio_err = BIO_new_fp(stderr, BIO_NOCLOSE);
     
-    SSLeay_add_all_algorithms();
-    ERR_load_crypto_strings();
-    
     mkcert(&x509, &pkey, NUM_BITS, SERIAL, NUM_YEARS);
 
     p12 = PKCS12_create("limelight", "GameStream", pkey, x509, NULL, 0, 0, 0, 0, 0);
     if (p12 == NULL) {
         printf("Error generating a valid PKCS12 certificate.\n");
     }
-
-    // Debug Print statements
-    //RSA_print_fp(stdout, pkey->pkey.rsa, 0);
-    //X509_print_fp(stdout, x509);
-    //PEM_write_PUBKEY(stdout, pkey);
-    //PEM_write_PrivateKey(stdout, pkey, NULL, NULL, 0, NULL, NULL);
-    //PEM_write_X509(stdout, x509);
     
-#ifndef OPENSSL_NO_ENGINE
-    ENGINE_cleanup();
-#endif
-    CRYPTO_cleanup_all_ex_data();
-    
-    CRYPTO_mem_leaks(bio_err);
     BIO_free(bio_err);
     
     return (CertKeyPair){x509, pkey, p12};
@@ -61,92 +85,3 @@ void freeCertKeyPair(struct CertKeyPair certKeyPair) {
     EVP_PKEY_free(certKeyPair.pkey);
     PKCS12_free(certKeyPair.p12);
 }
-
-int mkcert(X509 **x509p, EVP_PKEY **pkeyp, int bits, int serial, int years) {
-    X509 *x;
-    EVP_PKEY *pk;
-    RSA *rsa;
-    X509_NAME *name = NULL;
-    
-    if (*pkeyp == NULL) {
-        if ((pk=EVP_PKEY_new()) == NULL) {
-            return(0);
-        }
-    } else {
-        pk = *pkeyp;
-    }
-    
-    if (*x509p == NULL) {
-        if ((x = X509_new()) == NULL) {
-            goto err;
-        }
-    } else {
-        x = *x509p;
-    }
-    
-    rsa = RSA_generate_key(bits, RSA_F4, NULL, NULL);
-    if (!EVP_PKEY_assign_RSA(pk, rsa)) {
-        goto err;
-    }
-    
-    X509_set_version(x, 2);
-    ASN1_INTEGER_set(X509_get_serialNumber(x), serial);
-    X509_gmtime_adj(X509_get_notBefore(x), 0);
-    X509_gmtime_adj(X509_get_notAfter(x), (long)60*60*24*365*years);
-    X509_set_pubkey(x, pk);
-    
-    name = X509_get_subject_name(x);
-    
-    /* This function creates and adds the entry, working out the
-     * correct string type and performing checks on its length.
-     */
-    X509_NAME_add_entry_by_txt(name,"CN", MBSTRING_ASC, (unsigned char*)"NVIDIA GameStream Client", -1, -1, 0);
-    
-    /* Its self signed so set the issuer name to be the same as the
-     * subject.
-     */
-    X509_set_issuer_name(x, name);
-    
-    /* Add various extensions: standard extensions */
-    add_ext(x, NID_basic_constraints, "critical,CA:TRUE");
-    add_ext(x, NID_key_usage, "critical,keyCertSign,cRLSign");
-    
-    add_ext(x, NID_subject_key_identifier, "hash");
-    
-    if (!X509_sign(x, pk, EVP_sha1())) {
-        goto err;
-    }
-    
-    *x509p = x;
-    *pkeyp = pk;
-    
-    return(1);
-err:
-    return(0);
-}
-
-/* Add extension using V3 code: we can set the config file as NULL
- * because we wont reference any other sections.
- */
-
-int add_ext(X509 *cert, int nid, char *value)
-{
-    X509_EXTENSION *ex;
-    X509V3_CTX ctx;
-    /* This sets the 'context' of the extensions. */
-    /* No configuration database */
-    X509V3_set_ctx_nodb(&ctx);
-    /* Issuer and subject certs: both the target since it is self signed,
-     * no request and no CRL
-     */
-    X509V3_set_ctx(&ctx, cert, cert, NULL, NULL, 0);
-    ex = X509V3_EXT_conf_nid(NULL, &ctx, nid, value);
-    if (!ex) {
-        return 0;
-    }
-    
-    X509_add_ext(cert, ex, -1);
-    X509_EXTENSION_free(ex);
-    return 1;
-}
-

Limelight/Database/DataManager.h

@@ -18,12 +18,14 @@
                           height:(NSInteger)height
                            width:(NSInteger)width
                 onscreenControls:(NSInteger)onscreenControls
-                          remote:(BOOL)streamingRemotely
                    optimizeGames:(BOOL)optimizeGames
                  multiController:(BOOL)multiController
                        audioOnPC:(BOOL)audioOnPC
                          useHevc:(BOOL)useHevc
-                       enableHdr:(BOOL)enableHdr;
+                       enableHdr:(BOOL)enableHdr
+                  btMouseSupport:(BOOL)btMouseSupport
+               absoluteTouchMode:(BOOL)absoluteTouchMode
+                    statsOverlay:(BOOL)statsOverlay;
 
 - (NSArray*) getHosts;
 - (void) updateHost:(TemporaryHost*)host;

Limelight/Database/DataManager.m

@@ -21,12 +21,11 @@
     // HACK: Avoid calling [UIApplication delegate] off the UI thread to keep
     // Main Thread Checker happy.
     if ([NSThread isMainThread]) {
-        _appDelegate = (AppDelegate *)[[OSApplication sharedApplication] delegate];
-        
+        _appDelegate = (AppDelegate *)[[UIApplication sharedApplication] delegate];
     }
     else {
         dispatch_sync(dispatch_get_main_queue(), ^{
-            self->_appDelegate = (AppDelegate *)[[OSApplication sharedApplication] delegate];
+            self->_appDelegate = (AppDelegate *)[[UIApplication sharedApplication] delegate];
         });
     }
     
@@ -58,12 +57,14 @@
                           height:(NSInteger)height
                            width:(NSInteger)width
                 onscreenControls:(NSInteger)onscreenControls
-                          remote:(BOOL)streamingRemotely
                    optimizeGames:(BOOL)optimizeGames
                  multiController:(BOOL)multiController
                        audioOnPC:(BOOL)audioOnPC
                          useHevc:(BOOL)useHevc
-                       enableHdr:(BOOL)enableHdr {
+                       enableHdr:(BOOL)enableHdr
+                  btMouseSupport:(BOOL)btMouseSupport
+               absoluteTouchMode:(BOOL)absoluteTouchMode
+                    statsOverlay:(BOOL)statsOverlay {
     
     [_managedObjectContext performBlockAndWait:^{
         Settings* settingsToSave = [self retrieveSettings];
@@ -72,12 +73,14 @@
         settingsToSave.height = [NSNumber numberWithInteger:height];
         settingsToSave.width = [NSNumber numberWithInteger:width];
         settingsToSave.onscreenControls = [NSNumber numberWithInteger:onscreenControls];
-        settingsToSave.streamingRemotely = streamingRemotely;
         settingsToSave.optimizeGames = optimizeGames;
         settingsToSave.multiController = multiController;
         settingsToSave.playAudioOnPC = audioOnPC;
         settingsToSave.useHevc = useHevc;
         settingsToSave.enableHdr = enableHdr;
+        settingsToSave.btMouseSupport = btMouseSupport;
+        settingsToSave.absoluteTouchMode = absoluteTouchMode;
+        settingsToSave.statsOverlay = statsOverlay;
         
         [self saveData];
     }];

Limelight/Database/TemporaryApp.h

@@ -13,7 +13,9 @@
 
 @property (nullable, nonatomic, retain) NSString *id;
 @property (nullable, nonatomic, retain) NSString *name;
+@property (nullable, nonatomic, retain) NSString *installPath;
 @property (nonatomic)                   BOOL hdrSupported;
+@property (nonatomic)                   BOOL hidden;
 @property (nullable, nonatomic, retain) TemporaryHost *host;
 
 NS_ASSUME_NONNULL_BEGIN

Limelight/Database/TemporaryApp.m

@@ -16,6 +16,7 @@
     self.id = app.id;
     self.name = app.name;
     self.hdrSupported = app.hdrSupported;
+    self.hidden = app.hidden;
     self.host = tempHost;
     
     return self;
@@ -25,6 +26,7 @@
     parent.id = self.id;
     parent.name = self.name;
     parent.hdrSupported = self.hdrSupported;
+    parent.hidden = self.hidden;
     parent.host = host;
 }
 
@@ -41,7 +43,7 @@
         return YES;
     }
     
-    if (![object isKindOfClass:[App class]]) {
+    if (![object isKindOfClass:[self class]]) {
         return NO;
     }
     

Limelight/Database/TemporaryHost.h

@@ -11,23 +11,24 @@
 
 @interface TemporaryHost : NSObject
 
-@property (nonatomic) BOOL online;
-@property (nonatomic) PairState pairState;
-@property (nonatomic, nullable) NSString * activeAddress;
-@property (nonatomic, nullable) NSString * currentGame;
+@property (atomic) State state;
+@property (atomic) PairState pairState;
+@property (atomic, nullable, retain) NSString * activeAddress;
+@property (atomic, nullable, retain) NSString * currentGame;
 
-@property (nonatomic, nullable, retain) NSData *serverCert;
-@property (nonatomic, nullable, retain) NSString *address;
-@property (nonatomic, nullable, retain) NSString *externalAddress;
-@property (nonatomic, nullable, retain) NSString *localAddress;
-@property (nonatomic, nullable, retain) NSString *mac;
-@property (nonatomic)         int serverCodecModeSupport;
+@property (atomic, nullable, retain) NSData *serverCert;
+@property (atomic, nullable, retain) NSString *address;
+@property (atomic, nullable, retain) NSString *externalAddress;
+@property (atomic, nullable, retain) NSString *localAddress;
+@property (atomic, nullable, retain) NSString *ipv6Address;
+@property (atomic, nullable, retain) NSString *mac;
+@property (atomic)                   int serverCodecModeSupport;
 
 NS_ASSUME_NONNULL_BEGIN
 
-@property (nonatomic, retain) NSString *name;
-@property (nonatomic, retain) NSString *uuid;
-@property (nonatomic, retain) NSMutableSet *appList;
+@property (atomic, retain) NSString *name;
+@property (atomic, retain) NSString *uuid;
+@property (atomic, retain) NSSet *appList;
 
 - (id) initFromHost:(Host*)host;
 

Limelight/Database/TemporaryHost.m

@@ -16,6 +16,7 @@
     self = [super init];
     self.appList = [[NSMutableSet alloc] init];
     self.currentGame = @"0";
+    self.state = StateUnknown;
     
     return self;
 }
@@ -26,6 +27,7 @@
     self.address = host.address;
     self.externalAddress = host.externalAddress;
     self.localAddress = host.localAddress;
+    self.ipv6Address = host.ipv6Address;
     self.mac = host.mac;
     self.name = host.name;
     self.uuid = host.uuid;
@@ -60,6 +62,9 @@
     if (self.localAddress != nil) {
         parentHost.localAddress = self.localAddress;
     }
+    if (self.ipv6Address != nil) {
+        parentHost.ipv6Address = self.ipv6Address;
+    }
     if (self.mac != nil) {
         parentHost.mac = self.mac;
     }
@@ -85,7 +90,7 @@
         return YES;
     }
     
-    if (![object isKindOfClass:[Host class]]) {
+    if (![object isKindOfClass:[self class]]) {
         return NO;
     }
     

Limelight/Database/TemporarySettings.h

@@ -18,12 +18,14 @@
 @property (nonatomic, retain) NSNumber * width;
 @property (nonatomic, retain) NSNumber * onscreenControls;
 @property (nonatomic, retain) NSString * uniqueId;
-@property (nonatomic) BOOL streamingRemotely;
 @property (nonatomic) BOOL useHevc;
 @property (nonatomic) BOOL multiController;
 @property (nonatomic) BOOL playAudioOnPC;
 @property (nonatomic) BOOL optimizeGames;
 @property (nonatomic) BOOL enableHdr;
+@property (nonatomic) BOOL btMouseSupport;
+@property (nonatomic) BOOL absoluteTouchMode;
+@property (nonatomic) BOOL statsOverlay;
 
 - (id) initFromSettings:(Settings*)settings;
 

Limelight/Database/TemporarySettings.m

@@ -17,26 +17,30 @@
     self.parent = settings;
     
 #if TARGET_OS_TV
-    NSInteger _bitrate = [[NSUserDefaults standardUserDefaults] integerForKey:@"bitrate"];
-    NSInteger _framerate = [[NSUserDefaults standardUserDefaults] integerForKey:@"framerate"];
-
-    if (_bitrate) {
-        self.bitrate = [NSNumber numberWithInteger:_bitrate];
-    } else {
-        self.bitrate = [NSNumber numberWithInteger:20000];
+    // Apply default values from our Root.plist
+    NSString* settingsBundle = [[NSBundle mainBundle] pathForResource:@"Settings" ofType:@"bundle"];
+    NSDictionary* settingsData = [NSDictionary dictionaryWithContentsOfFile:[settingsBundle stringByAppendingPathComponent:@"Root.plist"]];
+    NSArray* preferences = [settingsData objectForKey:@"PreferenceSpecifiers"];
+    NSMutableDictionary* defaultsToRegister = [[NSMutableDictionary alloc] initWithCapacity:[preferences count]];
+    for (NSDictionary* prefSpecification in preferences) {
+        NSString* key = [prefSpecification objectForKey:@"Key"];
+        if (key != nil) {
+            [defaultsToRegister setObject:[prefSpecification objectForKey:@"DefaultValue"] forKey:key];
+        }
     }
-
-    if (_framerate) {
-        self.framerate = [NSNumber numberWithInteger:_framerate];
-    } else {
-        self.framerate = [NSNumber numberWithInteger:60];
-    }
-
-    self.useHevc = [[NSUserDefaults standardUserDefaults] boolForKey:@"useHevc"] || NO;
-    self.playAudioOnPC = [[NSUserDefaults standardUserDefaults] boolForKey:@"audioOnPC"] || NO;
-    self.enableHdr = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableHdr"] || NO;
-    self.optimizeGames = [[NSUserDefaults standardUserDefaults] boolForKey:@"optimizeGames"] || YES;
-    self.multiController = [[NSUserDefaults standardUserDefaults] boolForKey:@"multipleControllers"] || YES;
+    [[NSUserDefaults standardUserDefaults] registerDefaults:defaultsToRegister];
+    
+    self.bitrate = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"bitrate"]];
+    assert([self.bitrate intValue] != 0);
+    self.framerate = [NSNumber numberWithInteger:[[NSUserDefaults standardUserDefaults] integerForKey:@"framerate"]];
+    assert([self.framerate intValue] != 0);
+    self.useHevc = [[NSUserDefaults standardUserDefaults] boolForKey:@"useHevc"];
+    self.playAudioOnPC = [[NSUserDefaults standardUserDefaults] boolForKey:@"audioOnPC"];
+    self.enableHdr = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableHdr"];
+    self.optimizeGames = [[NSUserDefaults standardUserDefaults] boolForKey:@"optimizeGames"];
+    self.multiController = [[NSUserDefaults standardUserDefaults] boolForKey:@"multipleControllers"];
+    self.btMouseSupport = [[NSUserDefaults standardUserDefaults] boolForKey:@"btMouseSupport"];
+    self.statsOverlay = [[NSUserDefaults standardUserDefaults] boolForKey:@"statsOverlay"];
     
     NSInteger _screenSize = [[NSUserDefaults standardUserDefaults] integerForKey:@"streamResolution"];
     switch (_screenSize) {
@@ -44,17 +48,18 @@
             self.height = [NSNumber numberWithInteger:720];
             self.width = [NSNumber numberWithInteger:1280];
             break;
+        case 1:
+            self.height = [NSNumber numberWithInteger:1080];
+            self.width = [NSNumber numberWithInteger:1920];
+            break;
         case 2:
             self.height = [NSNumber numberWithInteger:2160];
             self.width = [NSNumber numberWithInteger:3840];
             break;
-        case 1:
         default:
-            self.height = [NSNumber numberWithInteger:1080];
-            self.width = [NSNumber numberWithInteger:1920];
-            break;
+            abort();
     }
-    self.onscreenControls = (NSInteger)OnScreenControlsLevelOff;
+    self.onscreenControls = [NSNumber numberWithInteger:OnScreenControlsLevelOff];
 #else
     self.bitrate = settings.bitrate;
     self.framerate = settings.framerate;
@@ -66,9 +71,11 @@
     self.optimizeGames = settings.optimizeGames;
     self.multiController = settings.multiController;
     self.onscreenControls = settings.onscreenControls;
+    self.btMouseSupport = settings.btMouseSupport;
+    self.absoluteTouchMode = settings.absoluteTouchMode;
+    self.statsOverlay = settings.statsOverlay;
 #endif
     self.uniqueId = settings.uniqueId;
-    self.streamingRemotely = settings.streamingRemotely;
     
     return self;
 }

Limelight/Images.xcassets/AddComputerIcon.imageset/Contents.json

@@ -1,23 +0,0 @@
-{
-  "images" : [
-    {
-      "idiom" : "universal",
-      "scale" : "1x",
-      "filename" : "limelight_computer_add_icon_1x.png"
-    },
-    {
-      "idiom" : "universal",
-      "scale" : "2x",
-      "filename" : "limelight_computer_add_icon_2x.png"
-    },
-    {
-      "idiom" : "universal",
-      "scale" : "3x",
-      "filename" : "limelight_computer_add_icon_3x.png"
-    }
-  ],
-  "info" : {
-    "version" : 1,
-    "author" : "xcode"
-  }
-}

Limelight/Images.xcassets/AddOverlayIcon.imageset/Contents.json

@@ -2,11 +2,11 @@
   "images" : [
     {
       "idiom" : "universal",
+      "filename" : "add.pdf",
       "scale" : "1x"
     },
     {
       "idiom" : "universal",
-      "filename" : "logo.png",
       "scale" : "2x"
     },
     {

Limelight/Images.xcassets/AddOverlayIcon.imageset/add.pdf

@@ -0,0 +1,68 @@
+%PDF-1.5
+%µí®û
+4 0 obj
+<< /Length 5 0 R
+   /Filter /FlateDecode
+>>
+stream
+xœ]P[à ûÏ)r�1}°cìÒº�öcÛý¥AèZ!%q였°/ç"%Èè9oô&áz>_ž—/%qaä!ºyä�§T3ÐjqµîaÚ”t„,~Dû›ÖÚGHýôzQõ␍®î:xgµ†I6Âß:gP((	hñ¯-1Îü5綎n3»!&úÖ"˜B-Õ¨~1–@½ZÒ¹](¡¿d÷ë9ا-žt§å…aL
+endstream
+endobj
+5 0 obj
+   183
+endobj
+3 0 obj
+<<
+   /ExtGState <<
+      /a0 << /CA 1 /ca 1 >>
+   >>
+>>
+endobj
+2 0 obj
+<< /Type /Page % 1
+   /Parent 1 0 R
+   /MediaBox [ 0 0 150 150 ]
+   /Contents 4 0 R
+   /Group <<
+      /Type /Group
+      /S /Transparency
+      /I true
+      /CS /DeviceRGB
+   >>
+   /Resources 3 0 R
+>>
+endobj
+1 0 obj
+<< /Type /Pages
+   /Kids [ 2 0 R ]
+   /Count 1
+>>
+endobj
+6 0 obj
+<< /Producer (cairo 1.16.0 (https://cairographics.org))
+   /CreationDate (D:20190922201812-07'00)
+>>
+endobj
+7 0 obj
+<< /Type /Catalog
+   /Pages 1 0 R
+>>
+endobj
+xref
+0 8
+0000000000 65535 f 
+0000000587 00000 n 
+0000000369 00000 n 
+0000000297 00000 n 
+0000000015 00000 n 
+0000000275 00000 n 
+0000000652 00000 n 
+0000000768 00000 n 
+trailer
+<< /Size 8
+   /Root 7 0 R
+   /Info 6 0 R
+>>
+startxref
+820
+%%EOF

Limelight/Images.xcassets/Computer.imageset/Computer.pdf

@@ -0,0 +1,71 @@
+%PDF-1.5
+%µí®û
+4 0 obj
+<< /Length 5 0 R
+   /Filter /FlateDecode
+>>
+stream
+xœ]‘[ƒ Eÿ³Šl )
y¸Œ.¡ÃTû¡µûŸi€ Õa9×›�ÑȺ±l.zÌ+|ÀTm›ñþ48a”ä_°äÑ&&ë‘SØ^8
cYÕÍÅíl"‹«Æ5ºá‚v¤�’"{
+ò<9¶�j�­7Ö!Db	×,Ú�–PÊbåÐjt¤•øzÅÚœÔ:ƒº,'á½â‚ýÒ‡ wRÁù‘Ê|µ8)’w
+-{GÍÑsµ™�æüÞG^’íosû»£MÅkäø
à=h_
+endstream
+endobj
+5 0 obj
+   214
+endobj
+3 0 obj
+<<
+   /ExtGState <<
+      /a0 << /CA 0.3 /ca 0.3 >>
+      /a1 << /CA 1 /ca 1 >>
+   >>
+>>
+endobj
+2 0 obj
+<< /Type /Page % 1
+   /Parent 1 0 R
+   /MediaBox [ 0 0 375 375 ]
+   /Contents 4 0 R
+   /Group <<
+      /Type /Group
+      /S /Transparency
+      /I true
+      /CS /DeviceRGB
+   >>
+   /Resources 3 0 R
+>>
+endobj
+1 0 obj
+<< /Type /Pages
+   /Kids [ 2 0 R ]
+   /Count 1
+>>
+endobj
+6 0 obj
+<< /Producer (cairo 1.16.0 (https://cairographics.org))
+   /CreationDate (D:20190922201805-07'00)
+>>
+endobj
+7 0 obj
+<< /Type /Catalog
+   /Pages 1 0 R
+>>
+endobj
+xref
+0 8
+0000000000 65535 f 
+0000000650 00000 n 
+0000000432 00000 n 
+0000000328 00000 n 
+0000000015 00000 n 
+0000000306 00000 n 
+0000000715 00000 n 
+0000000831 00000 n 
+trailer
+<< /Size 8
+   /Root 7 0 R
+   /Info 6 0 R
+>>
+startxref
+883
+%%EOF

Limelight/Images.xcassets/Computer.imageset/Contents.json

@@ -2,18 +2,16 @@
   "images" : [
     {
       "idiom" : "universal",
-      "scale" : "1x",
-      "filename" : "limelight_computer_1x.png"
+      "filename" : "Computer.pdf",
+      "scale" : "1x"
     },
     {
       "idiom" : "universal",
-      "scale" : "2x",
-      "filename" : "limelight_computer_2x.png"
+      "scale" : "2x"
     },
     {
       "idiom" : "universal",
-      "scale" : "3x",
-      "filename" : "limelight_computer_3x.png"
+      "scale" : "3x"
     }
   ],
   "info" : {

Limelight/Images.xcassets/ErrorOverlayIcon.imageset/Contents.json

@@ -2,17 +2,15 @@
   "images" : [
     {
       "idiom" : "universal",
-      "filename" : "pull-arrow-light-14x31@1x.png",
+      "filename" : "error.pdf",
       "scale" : "1x"
     },
     {
       "idiom" : "universal",
-      "filename" : "pull-arrow-light-28x61@2x.png",
       "scale" : "2x"
     },
     {
       "idiom" : "universal",
-      "filename" : "pull-arrow-light-42x92@3x.png",
       "scale" : "3x"
     }
   ],

Limelight/Images.xcassets/ErrorOverlayIcon.imageset/error.pdf

@@ -0,0 +1,68 @@
+%PDF-1.5
+%µí®û
+4 0 obj
+<< /Length 5 0 R
+   /Filter /FlateDecode
+>>
+stream
+xœUMA€ »ïý€¸� ßð	ÆD=ÈAý"L4š&[»µ›€3ÉE<cJ´“ àXÐŽŒå¤`¬‡8)-A:gúWo(ÜšÂVŠ:Õ	!ª±n«bμú”?©ØB*žL¸ܶðÿUS3␍t
H(å
+endstream
+endobj
+5 0 obj
+   110
+endobj
+3 0 obj
+<<
+   /ExtGState <<
+      /a0 << /CA 1 /ca 1 >>
+   >>
+>>
+endobj
+2 0 obj
+<< /Type /Page % 1
+   /Parent 1 0 R
+   /MediaBox [ 0 0 150 150 ]
+   /Contents 4 0 R
+   /Group <<
+      /Type /Group
+      /S /Transparency
+      /I true
+      /CS /DeviceRGB
+   >>
+   /Resources 3 0 R
+>>
+endobj
+1 0 obj
+<< /Type /Pages
+   /Kids [ 2 0 R ]
+   /Count 1
+>>
+endobj
+6 0 obj
+<< /Producer (cairo 1.16.0 (https://cairographics.org))
+   /CreationDate (D:20190922201819-07'00)
+>>
+endobj
+7 0 obj
+<< /Type /Catalog
+   /Pages 1 0 R
+>>
+endobj
+xref
+0 8
+0000000000 65535 f 
+0000000514 00000 n 
+0000000296 00000 n 
+0000000224 00000 n 
+0000000015 00000 n 
+0000000202 00000 n 
+0000000579 00000 n 
+0000000695 00000 n 
+trailer
+<< /Size 8
+   /Root 7 0 R
+   /Info 6 0 R
+>>
+startxref
+747
+%%EOF

Limelight/Images.xcassets/LockedOverlayIcon.imageset/Contents.json

@@ -0,0 +1,21 @@
+{
+  "images" : [
+    {
+      "idiom" : "universal",
+      "filename" : "lock.pdf",
+      "scale" : "1x"
+    },
+    {
+      "idiom" : "universal",
+      "scale" : "2x"
+    },
+    {
+      "idiom" : "universal",
+      "scale" : "3x"
+    }
+  ],
+  "info" : {
+    "version" : 1,
+    "author" : "xcode"
+  }
+}

Limelight/Images.xcassets/UpIcon.imageset/Contents.json

@@ -0,0 +1,21 @@
+{
+  "images" : [
+    {
+      "idiom" : "universal",
+      "filename" : "up_arrow.pdf",
+      "scale" : "1x"
+    },
+    {
+      "idiom" : "universal",
+      "scale" : "2x"
+    },
+    {
+      "idiom" : "universal",
+      "scale" : "3x"
+    }
+  ],
+  "info" : {
+    "version" : 1,
+    "author" : "xcode"
+  }
+}

Limelight/Images.xcassets/UpIcon.imageset/up_arrow.pdf

@@ -0,0 +1,70 @@
+%PDF-1.5
+%µí®û
+4 0 obj
+<< /Length 5 0 R
+   /Filter /FlateDecode
+>>
+stream
+xœUŽQ
+ƒ@DÿsŠ¹€1ÙÍF=†G(…êÇú¡½?-e`ƒÌ$
+	5Ö¼ZIqh›Ð¾Ó—4qAî¨ó ÓsTdã¹dl>üØ�H…	{º‰n|–KÏâé,Ë}°?‡ê5ôª™ÿ^øÐH;U$}
+endstream
+endobj
+5 0 obj
+   114
+endobj
+3 0 obj
+<<
+   /ExtGState <<
+      /a0 << /CA 1 /ca 1 >>
+   >>
+>>
+endobj
+2 0 obj
+<< /Type /Page % 1
+   /Parent 1 0 R
+   /MediaBox [ 0 0 75 75 ]
+   /Contents 4 0 R
+   /Group <<
+      /Type /Group
+      /S /Transparency
+      /I true
+      /CS /DeviceRGB
+   >>
+   /Resources 3 0 R
+>>
+endobj
+1 0 obj
+<< /Type /Pages
+   /Kids [ 2 0 R ]
+   /Count 1
+>>
+endobj
+6 0 obj
+<< /Producer (cairo 1.16.0 (https://cairographics.org))
+   /CreationDate (D:20190830203401-07'00)
+>>
+endobj
+7 0 obj
+<< /Type /Catalog
+   /Pages 1 0 R
+>>
+endobj
+xref
+0 8
+0000000000 65535 f 
+0000000516 00000 n 
+0000000300 00000 n 
+0000000228 00000 n 
+0000000015 00000 n 
+0000000206 00000 n 
+0000000581 00000 n 
+0000000697 00000 n 
+trailer
+<< /Size 8
+   /Root 7 0 R
+   /Info 6 0 R
+>>
+startxref
+749
+%%EOF

Limelight/Input/AbsoluteTouchHandler.h

@@ -0,0 +1,19 @@
+//
+//  AbsoluteTouchHandler.h
+//  Moonlight
+//
+//  Created by Cameron Gutman on 11/1/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+#import "StreamView.h"
+
+NS_ASSUME_NONNULL_BEGIN
+
+@interface AbsoluteTouchHandler : UIResponder
+
+-(id)initWithView:(StreamView*)view;
+
+@end
+
+NS_ASSUME_NONNULL_END

Limelight/Input/AbsoluteTouchHandler.m

@@ -0,0 +1,120 @@
+//
+//  AbsoluteTouchHandler.m
+//  Moonlight
+//
+//  Created by Cameron Gutman on 11/1/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+#import "AbsoluteTouchHandler.h"
+
+#include <Limelight.h>
+
+// How long the fingers must be stationary to start a right click
+#define LONG_PRESS_ACTIVATION_DELAY 0.650f
+
+// How far the finger can move before it cancels a right click
+#define LONG_PRESS_ACTIVATION_DELTA 0.01f
+
+// How long the double tap deadzone stays in effect between touch up and touch down
+#define DOUBLE_TAP_DEAD_ZONE_DELAY 0.250f
+
+// How far the finger can move before it can override the double tap deadzone
+#define DOUBLE_TAP_DEAD_ZONE_DELTA 0.025f
+
+@implementation AbsoluteTouchHandler {
+    StreamView* view;
+    
+    NSTimer* longPressTimer;
+    UITouch* lastTouchDown;
+    CGPoint lastTouchDownLocation;
+    UITouch* lastTouchUp;
+    CGPoint lastTouchUpLocation;
+}
+
+- (id)initWithView:(StreamView*)view {
+    self = [self init];
+    self->view = view;
+    return self;
+}
+
+- (void)onLongPressStart:(NSTimer*)timer {
+    // Raise the left click and start a right click
+    LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_RIGHT);
+}
+
+- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
+    // Ignore touch down events with more than one finger
+    if ([[event allTouches] count] > 1) {
+        return;
+    }
+    
+    UITouch* touch = [touches anyObject];
+    CGPoint touchLocation = [touch locationInView:view];
+    
+    // Don't reposition for finger down events within the deadzone. This makes double-clicking easier.
+    if (touch.timestamp - lastTouchUp.timestamp > DOUBLE_TAP_DEAD_ZONE_DELAY ||
+        sqrt(pow((touchLocation.x / view.bounds.size.width) - (lastTouchUpLocation.x / view.bounds.size.width), 2) +
+             pow((touchLocation.y / view.bounds.size.height) - (lastTouchUpLocation.y / view.bounds.size.height), 2)) > DOUBLE_TAP_DEAD_ZONE_DELTA) {
+        [view updateCursorLocation:touchLocation isMouse:NO];
+    }
+    
+    // Press the left button down
+    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+    
+    // Start the long press timer
+    longPressTimer = [NSTimer scheduledTimerWithTimeInterval:LONG_PRESS_ACTIVATION_DELAY
+                                                      target:self
+                                                    selector:@selector(onLongPressStart:)
+                                                    userInfo:nil
+                                                     repeats:NO];
+    
+    lastTouchDown = touch;
+    lastTouchDownLocation = touchLocation;
+}
+
+- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
+    // Ignore touch move events with more than one finger
+    if ([[event allTouches] count] > 1) {
+        return;
+    }
+    
+    UITouch* touch = [touches anyObject];
+    CGPoint touchLocation = [touch locationInView:view];
+    
+    if (sqrt(pow((touchLocation.x / view.bounds.size.width) - (lastTouchDownLocation.x / view.bounds.size.width), 2) +
+             pow((touchLocation.y / view.bounds.size.height) - (lastTouchDownLocation.y / view.bounds.size.height), 2)) > LONG_PRESS_ACTIVATION_DELTA) {
+        // Moved too far since touch down. Cancel the long press timer.
+        [longPressTimer invalidate];
+        longPressTimer = nil;
+    }
+    
+    [view updateCursorLocation:[[touches anyObject] locationInView:view] isMouse:NO];
+}
+
+- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
+    // Only fire this logic if all touches have ended
+    if ([[event allTouches] count] == [touches count]) {
+        // Cancel the long press timer
+        [longPressTimer invalidate];
+        longPressTimer = nil;
+
+        // Left button up on finger up
+        LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+
+        // Raise right button too in case we triggered a long press gesture
+        LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_RIGHT);
+        
+        // Remember this last touch for touch-down deadzoning
+        lastTouchUp = [touches anyObject];
+        lastTouchUpLocation = [lastTouchUp locationInView:view];
+    }
+}
+
+- (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
+    // Treat this as a normal touchesEnded event
+    [self touchesEnded:touches withEvent:event];
+}
+
+@end

Limelight/Input/Controller.h

@@ -6,7 +6,10 @@
 //  Copyright © 2019 Moonlight Game Streaming Project. All rights reserved.
 //
 
+#import "HapticContext.h"
+
 @import GameController;
+@import CoreHaptics;
 
 @interface Controller : NSObject
 
@@ -14,13 +17,15 @@
 @property (nonatomic)                   int playerIndex;
 @property (nonatomic)                   int lastButtonFlags;
 @property (nonatomic)                   int emulatingButtonFlags;
+@property (nonatomic)                   int supportedEmulationFlags;
 @property (nonatomic)                   unsigned char lastLeftTrigger;
 @property (nonatomic)                   unsigned char lastRightTrigger;
 @property (nonatomic)                   short lastLeftStickX;
 @property (nonatomic)                   short lastLeftStickY;
 @property (nonatomic)                   short lastRightStickX;
 @property (nonatomic)                   short lastRightStickY;
-@property (nonatomic)                   unsigned short lowFreqMotor;
-@property (nonatomic)                   unsigned short highFreqMotor;
+
+@property (nonatomic)                   HapticContext* _Nullable lowFreqMotor;
+@property (nonatomic)                   HapticContext* _Nullable highFreqMotor;
 
 @end

Limelight/Input/ControllerSupport.h

@@ -6,35 +6,32 @@
 //  Copyright (c) 2014 Moonlight Stream. All rights reserved.
 //
 
-// Swift
-#if !TARGET_OS_IPHONE
-#import "Gamepad.h"
-#endif
 #import "StreamConfiguration.h"
 #import "Controller.h"
 
 @class OnScreenControls;
 
+@protocol InputPresenceDelegate <NSObject>
+
+- (void) gamepadPresenceChanged;
+- (void) mousePresenceChanged;
+
+@end
+
 @interface ControllerSupport : NSObject
 
--(id) initWithConfig:(StreamConfiguration*)streamConfig;
+-(id) initWithConfig:(StreamConfiguration*)streamConfig presenceDelegate:(id<InputPresenceDelegate>)delegate;
 
-#if TARGET_OS_IPHONE
 -(void) initAutoOnScreenControlMode:(OnScreenControls*)osc;
 -(void) cleanup;
 -(Controller*) getOscController;
-#else
--(void) assignGamepad:(struct Gamepad_device *)gamepad;
--(void) removeGamepad:(struct Gamepad_device *)gamepad;
--(NSMutableDictionary*) getControllers;
-#endif
 
 -(void) updateLeftStick:(Controller*)controller x:(short)x y:(short)y;
 -(void) updateRightStick:(Controller*)controller x:(short)x y:(short)y;
 
--(void) updateLeftTrigger:(Controller*)controller left:(char)left;
--(void) updateRightTrigger:(Controller*)controller right:(char)right;
--(void) updateTriggers:(Controller*)controller left:(char)left right:(char)right;
+-(void) updateLeftTrigger:(Controller*)controller left:(unsigned char)left;
+-(void) updateRightTrigger:(Controller*)controller right:(unsigned char)right;
+-(void) updateTriggers:(Controller*)controller left:(unsigned char)left right:(unsigned char)right;
 
 -(void) updateButtonFlags:(Controller*)controller flags:(int)flags;
 -(void) setButtonFlag:(Controller*)controller flags:(int)flags;
@@ -46,7 +43,6 @@
 
 +(int) getConnectedGamepadMask:(StreamConfiguration*)streamConfig;
 
-@property (nonatomic, strong) id connectObserver;
-@property (nonatomic, strong) id disconnectObserver;
+-(NSUInteger) getConnectedGamepadCount;
 
 @end

Limelight/Input/ControllerSupport.m

@@ -10,10 +10,6 @@
 #import "Controller.h"
 
 #import "OnScreenControls.h"
-#if !TARGET_OS_IPHONE
-#import "Gamepad.h"
-#import "Control.h"
-#endif
 
 #import "DataManager.h"
 #include "Limelight.h"
@@ -21,10 +17,23 @@
 @import GameController;
 @import AudioToolbox;
 
+static const double MOUSE_SPEED_DIVISOR = 2.5;
+
 @implementation ControllerSupport {
+    id _controllerConnectObserver;
+    id _controllerDisconnectObserver;
+    id _mouseConnectObserver;
+    id _mouseDisconnectObserver;
+    id _keyboardConnectObserver;
+    id _keyboardDisconnectObserver;
+    
     NSLock *_controllerStreamLock;
     NSMutableDictionary *_controllers;
-    NSTimer *_rumbleTimer;
+    id<InputPresenceDelegate> _presenceDelegate;
+    
+    float accumulatedDeltaX;
+    float accumulatedDeltaY;
+    float accumulatedScrollY;
     
     OnScreenControls *_osc;
     
@@ -44,18 +53,6 @@
 #define UPDATE_BUTTON_FLAG(controller, x, y) \
 ((y) ? [self setButtonFlag:controller flags:x] : [self clearButtonFlag:controller flags:x])
 
--(void) rumbleController: (Controller*)controller
-{
-    // Only vibrate if the amplitude is large enough
-    if (controller.lowFreqMotor > 0x5000 || controller.highFreqMotor > 0x5000) {
-        // If the gamepad is nil (on-screen controls) or it's attached to the device,
-        // then vibrate the device itself
-        if (controller.gamepad == nil || [controller.gamepad isAttachedToDevice]) {
-            AudioServicesPlayAlertSound(kSystemSoundID_Vibrate);
-        }
-    }
-}
-
 -(void) rumble:(unsigned short)controllerNumber lowFreqMotor:(unsigned short)lowFreqMotor highFreqMotor:(unsigned short)highFreqMotor
 {
     Controller* controller = [_controllers objectForKey:[NSNumber numberWithInteger:controllerNumber]];
@@ -68,12 +65,8 @@
         return;
     }
     
-    // Update the motor levels for the rumble timer to grab next iteration
-    controller.lowFreqMotor = lowFreqMotor;
-    controller.highFreqMotor = highFreqMotor;
-    
-    // Rumble now to ensure short vibrations aren't missed
-    [self rumbleController:controller];
+    [controller.lowFreqMotor setMotorAmplitude:lowFreqMotor];
+    [controller.highFreqMotor setMotorAmplitude:highFreqMotor];
 }
 
 -(void) updateLeftStick:(Controller*)controller x:(short)x y:(short)y
@@ -92,21 +85,21 @@
     }
 }
 
--(void) updateLeftTrigger:(Controller*)controller left:(char)left
+-(void) updateLeftTrigger:(Controller*)controller left:(unsigned char)left
 {
     @synchronized(controller) {
         controller.lastLeftTrigger = left;
     }
 }
 
--(void) updateRightTrigger:(Controller*)controller right:(char)right
+-(void) updateRightTrigger:(Controller*)controller right:(unsigned char)right
 {
     @synchronized(controller) {
         controller.lastRightTrigger = right;
     }
 }
 
--(void) updateTriggers:(Controller*) controller left:(char)left right:(char)right
+-(void) updateTriggers:(Controller*) controller left:(unsigned char)left right:(unsigned char)right
 {
     @synchronized(controller) {
         controller.lastLeftTrigger = left;
@@ -116,17 +109,26 @@
 
 -(void) handleSpecialCombosReleased:(Controller*)controller releasedButtons:(int)releasedButtons
 {
-    if ((controller.emulatingButtonFlags & EMULATING_SELECT) &&
-        ((releasedButtons & LB_FLAG) || (releasedButtons & PLAY_FLAG))) {
+    if ((controller.emulatingButtonFlags & EMULATING_SELECT) && (releasedButtons & (LB_FLAG | PLAY_FLAG))) {
         controller.lastButtonFlags &= ~BACK_FLAG;
         controller.emulatingButtonFlags &= ~EMULATING_SELECT;
     }
-    if ((controller.emulatingButtonFlags & EMULATING_SPECIAL) &&
-        ((releasedButtons & RB_FLAG) || (releasedButtons & PLAY_FLAG) ||
-         (releasedButtons & BACK_FLAG))) {
-            controller.lastButtonFlags &= ~SPECIAL_FLAG;
-            controller.emulatingButtonFlags &= ~EMULATING_SPECIAL;
+    
+    if (controller.emulatingButtonFlags & EMULATING_SPECIAL) {
+        // If Select is emulated, we use RB+Start to emulate special, otherwise we use Start+Select
+        if (controller.supportedEmulationFlags & EMULATING_SELECT) {
+            if (releasedButtons & (RB_FLAG | PLAY_FLAG)) {
+                controller.lastButtonFlags &= ~SPECIAL_FLAG;
+                controller.emulatingButtonFlags &= ~EMULATING_SPECIAL;
+            }
         }
+        else {
+            if (releasedButtons & (BACK_FLAG | PLAY_FLAG)) {
+                controller.lastButtonFlags &= ~SPECIAL_FLAG;
+                controller.emulatingButtonFlags &= ~EMULATING_SPECIAL;
+            }
+        }
+    }
 }
 
 -(void) handleSpecialCombosPressed:(Controller*)controller pressedButtons:(int)pressedButtons
@@ -135,18 +137,31 @@
     if (controller.lastButtonFlags & PLAY_FLAG) {
         // If LB and start are down, trigger select
         if (controller.lastButtonFlags & LB_FLAG) {
-            controller.lastButtonFlags |= BACK_FLAG;
-            controller.lastButtonFlags &= ~(pressedButtons & (PLAY_FLAG | LB_FLAG));
-            controller.emulatingButtonFlags |= EMULATING_SELECT;
+            if (controller.supportedEmulationFlags & EMULATING_SELECT) {
+                controller.lastButtonFlags |= BACK_FLAG;
+                controller.lastButtonFlags &= ~(pressedButtons & (PLAY_FLAG | LB_FLAG));
+                controller.emulatingButtonFlags |= EMULATING_SELECT;
+            }
         }
-        // If (RB or select) and start are down, trigger special
-        else if ((controller.lastButtonFlags & RB_FLAG) || (controller.lastButtonFlags & BACK_FLAG)) {
-            controller.lastButtonFlags |= SPECIAL_FLAG;
-            controller.lastButtonFlags &= ~(pressedButtons & (PLAY_FLAG | RB_FLAG | BACK_FLAG));
-            controller.emulatingButtonFlags |= EMULATING_SPECIAL;
+        else if (controller.supportedEmulationFlags & EMULATING_SPECIAL) {
+            // If Select is emulated too, use RB+Start to emulate special
+            if (controller.supportedEmulationFlags & EMULATING_SELECT) {
+                if (controller.lastButtonFlags & RB_FLAG) {
+                    controller.lastButtonFlags |= SPECIAL_FLAG;
+                    controller.lastButtonFlags &= ~(pressedButtons & (PLAY_FLAG | RB_FLAG));
+                    controller.emulatingButtonFlags |= EMULATING_SPECIAL;
+                }
+            }
+            else {
+                // If Select is physical, use Start+Select to emulate special
+                if (controller.lastButtonFlags & BACK_FLAG) {
+                    controller.lastButtonFlags |= SPECIAL_FLAG;
+                    controller.lastButtonFlags &= ~(pressedButtons & (PLAY_FLAG | BACK_FLAG));
+                    controller.emulatingButtonFlags |= EMULATING_SPECIAL;
+                }
+            }
         }
     }
-    
 }
 
 -(void) updateButtonFlags:(Controller*)controller flags:(int)flags
@@ -192,12 +207,31 @@
     [_controllerStreamLock unlock];
 }
 
++(BOOL) hasKeyboardOrMouse {
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        return GCMouse.mice.count > 0 || GCKeyboard.coalescedKeyboard != nil;
+    }
+    else {
+        return NO;
+    }
+}
+
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wdeprecated-declarations"
+
 -(void) unregisterControllerCallbacks:(GCController*) controller
 {
     if (controller != NULL) {
         controller.controllerPausedHandler = NULL;
         
         if (controller.extendedGamepad != NULL) {
+            // Re-enable system gestures on the gamepad buttons now
+            if (@available(iOS 14.0, tvOS 14.0, *)) {
+                for (GCControllerElement* element in controller.extendedGamepad.allElements) {
+                    element.preferredSystemGestureState = GCSystemGestureStateEnabled;
+                }
+            }
+            
             controller.extendedGamepad.valueChangedHandler = NULL;
         }
         else if (controller.gamepad != NULL) {
@@ -206,31 +240,67 @@
     }
 }
 
+-(void) initializeControllerHaptics:(Controller*) controller
+{
+    controller.lowFreqMotor = [HapticContext createContextForLowFreqMotor:controller.gamepad];
+    controller.highFreqMotor = [HapticContext createContextForHighFreqMotor:controller.gamepad];
+}
+
+-(void) cleanupControllerHaptics:(Controller*) controller
+{
+    [controller.lowFreqMotor cleanup];
+    [controller.highFreqMotor cleanup];
+}
+
 -(void) registerControllerCallbacks:(GCController*) controller
 {
     if (controller != NULL) {
-        controller.controllerPausedHandler = ^(GCController *controller) {
-            Controller* limeController = [self->_controllers objectForKey:[NSNumber numberWithInteger:controller.playerIndex]];
-            
-            // Get off the main thread
-            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
-                [self setButtonFlag:limeController flags:PLAY_FLAG];
-                [self updateFinished:limeController];
+        // iOS 13 allows the Start button to behave like a normal button, however
+        // older MFi controllers can send an instant down+up event for the start button
+        // which means the button will not be down long enough to register on the PC.
+        // To work around this issue, use the old controllerPausedHandler if the controller
+        // doesn't have a Select button (which indicates it probably doesn't have a proper
+        // Start button either).
+        BOOL useLegacyPausedHandler = YES;
+        if (@available(iOS 13.0, tvOS 13.0, *)) {
+            if (controller.extendedGamepad != nil &&
+                controller.extendedGamepad.buttonOptions != nil) {
+                useLegacyPausedHandler = NO;
+            }
+        }
+        
+        if (useLegacyPausedHandler) {
+            controller.controllerPausedHandler = ^(GCController *controller) {
+                Controller* limeController = [self->_controllers objectForKey:[NSNumber numberWithInteger:controller.playerIndex]];
                 
-                // Pause for 100 ms
-                usleep(100 * 1000);
-                
-                [self clearButtonFlag:limeController flags:PLAY_FLAG];
-                [self updateFinished:limeController];
-            });
-        };
+                // Get off the main thread
+                dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
+                    [self setButtonFlag:limeController flags:PLAY_FLAG];
+                    [self updateFinished:limeController];
+                    
+                    // Pause for 100 ms
+                    usleep(100 * 1000);
+                    
+                    [self clearButtonFlag:limeController flags:PLAY_FLAG];
+                    [self updateFinished:limeController];
+                });
+            };
+        }
         
         if (controller.extendedGamepad != NULL) {
+            // Disable system gestures on the gamepad to avoid interfering
+            // with in-game controller actions
+            if (@available(iOS 14.0, tvOS 14.0, *)) {
+                for (GCControllerElement* element in controller.extendedGamepad.allElements) {
+                    element.preferredSystemGestureState = GCSystemGestureStateDisabled;
+                }
+            }
+            
             controller.extendedGamepad.valueChangedHandler = ^(GCExtendedGamepad *gamepad, GCControllerElement *element) {
                 Controller* limeController = [self->_controllers objectForKey:[NSNumber numberWithInteger:gamepad.controller.playerIndex]];
                 short leftStickX, leftStickY;
                 short rightStickX, rightStickY;
-                char leftTrigger, rightTrigger;
+                unsigned char leftTrigger, rightTrigger;
                 
                 UPDATE_BUTTON_FLAG(limeController, A_FLAG, gamepad.buttonA.pressed);
                 UPDATE_BUTTON_FLAG(limeController, B_FLAG, gamepad.buttonB.pressed);
@@ -255,6 +325,24 @@
                     }
                 }
                 
+                if (@available(iOS 13.0, tvOS 13.0, *)) {
+                    // Options button is optional (only present on Xbox One S and PS4 gamepads)
+                    if (gamepad.buttonOptions != nil) {
+                        UPDATE_BUTTON_FLAG(limeController, BACK_FLAG, gamepad.buttonOptions.pressed);
+
+                        // For older MFi gamepads, the menu button will already be handled by
+                        // the controllerPausedHandler.
+                        UPDATE_BUTTON_FLAG(limeController, PLAY_FLAG, gamepad.buttonMenu.pressed);
+                    }
+                }
+                
+                if (@available(iOS 14.0, tvOS 14.0, *)) {
+                    // Home/Guide button is optional (only present on Xbox One S and PS4 gamepads)
+                    if (gamepad.buttonHome != nil) {
+                        UPDATE_BUTTON_FLAG(limeController, SPECIAL_FLAG, gamepad.buttonHome.pressed);
+                    }
+                }
+                
                 leftStickX = gamepad.leftThumbstick.xAxis.value * 0x7FFE;
                 leftStickY = gamepad.leftThumbstick.yAxis.value * 0x7FFE;
                 
@@ -294,6 +382,76 @@
     }
 }
 
+-(void) unregisterMouseCallbacks:(GCMouse*)mouse API_AVAILABLE(ios(14.0)) {
+    mouse.mouseInput.mouseMovedHandler = nil;
+    
+    mouse.mouseInput.leftButton.pressedChangedHandler = nil;
+    mouse.mouseInput.middleButton.pressedChangedHandler = nil;
+    mouse.mouseInput.rightButton.pressedChangedHandler = nil;
+    
+    for (GCControllerButtonInput* auxButton in mouse.mouseInput.auxiliaryButtons) {
+        auxButton.pressedChangedHandler = nil;
+    }
+}
+
+-(void) registerMouseCallbacks:(GCMouse*) mouse API_AVAILABLE(ios(14.0)) {
+    mouse.mouseInput.mouseMovedHandler = ^(GCMouseInput * _Nonnull mouse, float deltaX, float deltaY) {
+        self->accumulatedDeltaX += deltaX / MOUSE_SPEED_DIVISOR;
+        self->accumulatedDeltaY += -deltaY / MOUSE_SPEED_DIVISOR;
+        
+        short truncatedDeltaX = (short)self->accumulatedDeltaX;
+        short truncatedDeltaY = (short)self->accumulatedDeltaY;
+        
+        if (truncatedDeltaX != 0 || truncatedDeltaY != 0) {
+            LiSendMouseMoveEvent(truncatedDeltaX, truncatedDeltaY);
+            
+            self->accumulatedDeltaX -= truncatedDeltaX;
+            self->accumulatedDeltaY -= truncatedDeltaY;
+        }
+    };
+    
+    mouse.mouseInput.leftButton.pressedChangedHandler = ^(GCControllerButtonInput * _Nonnull button, float value, BOOL pressed) {
+        LiSendMouseButtonEvent(pressed ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+    };
+    mouse.mouseInput.middleButton.pressedChangedHandler = ^(GCControllerButtonInput * _Nonnull button, float value, BOOL pressed) {
+        LiSendMouseButtonEvent(pressed ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, BUTTON_MIDDLE);
+    };
+    mouse.mouseInput.rightButton.pressedChangedHandler = ^(GCControllerButtonInput * _Nonnull button, float value, BOOL pressed) {
+        LiSendMouseButtonEvent(pressed ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, BUTTON_RIGHT);
+    };
+    
+    if (mouse.mouseInput.auxiliaryButtons != nil) {
+        if (mouse.mouseInput.auxiliaryButtons.count >= 1) {
+            mouse.mouseInput.auxiliaryButtons[0].pressedChangedHandler = ^(GCControllerButtonInput * _Nonnull button, float value, BOOL pressed) {
+                LiSendMouseButtonEvent(pressed ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, BUTTON_X1);
+            };
+        }
+        if (mouse.mouseInput.auxiliaryButtons.count >= 2) {
+            mouse.mouseInput.auxiliaryButtons[1].pressedChangedHandler = ^(GCControllerButtonInput * _Nonnull button, float value, BOOL pressed) {
+                LiSendMouseButtonEvent(pressed ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, BUTTON_X2);
+            };
+        }
+    }
+    
+    // We use UIPanGestureRecognizer on iPadOS because it allows us to distinguish
+    // between discrete and continuous scroll events and also works around a bug
+    // in iPadOS 15 where discrete scroll events are dropped. tvOS only supports
+    // GCMouse for mice, so we will have to just use it and hope for the best.
+#if TARGET_OS_TV
+    mouse.mouseInput.scroll.yAxis.valueChangedHandler = ^(GCControllerAxisInput * _Nonnull axis, float value) {
+        self->accumulatedScrollY += value;
+        
+        short truncatedScrollY = (short)self->accumulatedScrollY;
+        
+        if (truncatedScrollY != 0) {
+            LiSendHighResScrollEvent(truncatedScrollY * 20);
+            
+            self->accumulatedScrollY -= truncatedScrollY;
+        }
+    };
+#endif
+}
+
 -(void) updateAutoOnScreenControlMode
 {
     // Auto on-screen control support may not be enabled
@@ -315,6 +473,12 @@
                     if (controller.extendedGamepad.leftThumbstickButton != nil &&
                         controller.extendedGamepad.rightThumbstickButton != nil) {
                         level = OnScreenControlsLevelAutoGCExtendedGamepadWithStickButtons;
+                        if (@available(iOS 13.0, tvOS 13.0, *)) {
+                            if (controller.extendedGamepad.buttonOptions != nil) {
+                                // Has L3/R3 and Select, so we can show nothing :)
+                                level = OnScreenControlsLevelOff;
+                            }
+                        }
                     }
                 }
                 break;
@@ -326,9 +490,18 @@
         }
     }
     
-#if TARGET_OS_IPHONE
+    // If we didn't find a gamepad present and we have a keyboard or mouse, turn
+    // the on-screen controls off to get the overlays out of the way.
+    if (level == OnScreenControlsLevelFull && [ControllerSupport hasKeyboardOrMouse]) {
+        level = OnScreenControlsLevelOff;
+        
+        // Ensure the virtual gamepad disappears to avoid confusing some games.
+        // If the mouse and keyboard disconnect later, it will reappear when the
+        // first OSC input is received.
+        LiSendMultiControllerEvent(0, 0, 0, 0, 0, 0, 0, 0, 0);
+    }
+    
     [_osc setLevel:level];
-#endif
 }
 
 -(void) initAutoOnScreenControlMode:(OnScreenControls*)osc
@@ -354,7 +527,27 @@
                 limeController.playerIndex = i;
             }
             
+            limeController.supportedEmulationFlags = EMULATING_SPECIAL | EMULATING_SELECT;
             limeController.gamepad = controller;
+            
+            if (@available(iOS 13.0, tvOS 13.0, *)) {
+                if (controller.extendedGamepad != nil &&
+                    controller.extendedGamepad.buttonOptions != nil) {
+                    // Disable select button emulation since we have a physical select button
+                    limeController.supportedEmulationFlags &= ~EMULATING_SELECT;
+                }
+            }
+            
+            if (@available(iOS 14.0, tvOS 14.0, *)) {
+                if (controller.extendedGamepad != nil &&
+                    controller.extendedGamepad.buttonHome != nil) {
+                    // Disable special button emulation since we have a physical special button
+                    limeController.supportedEmulationFlags &= ~EMULATING_SPECIAL;
+                }
+            }
+            
+            // Prepare controller haptics for use
+            [self initializeControllerHaptics:limeController];
 
             [_controllers setObject:limeController forKey:[NSNumber numberWithInteger:controller.playerIndex]];
             
@@ -364,45 +557,16 @@
     }
 }
 
-#if TARGET_OS_IPHONE
 -(Controller*) getOscController {
     return _player0osc;
 }
-#else
--(NSMutableDictionary*) getControllers {
-    return _controllers;
-}
-
--(void) assignGamepad:(struct Gamepad_device *)gamepad {
-    for (int i = 0; i < 4; i++) {
-        if (!(_controllerNumbers & (1 << i))) {
-            _controllerNumbers |= (1 << i);
-            gamepad->deviceID = i;
-            NSLog(@"Gamepad device id: %u assigned", gamepad->deviceID);
-            Controller* limeController;
-            limeController = [[Controller alloc] init];
-            limeController.playerIndex = i;
-            
-            [_controllers setObject:limeController forKey:[NSNumber numberWithInteger:i]];
-            break;
-        }
-    }
-}
-
--(void) removeGamepad:(struct Gamepad_device *)gamepad {
-    _controllerNumbers &= ~(1 << gamepad->deviceID);
-    Log(LOG_I, @"Unassigning controller index: %ld", (long)gamepad->deviceID);
-    
-    // Inform the server of the updated active gamepads before removing this controller
-    [self updateFinished:[_controllers objectForKey:[NSNumber numberWithInteger:gamepad->deviceID]]];
-    [_controllers removeObjectForKey:[NSNumber numberWithInteger:gamepad->deviceID]];
-}
-#endif
 
 +(bool) isSupportedGamepad:(GCController*) controller {
     return controller.extendedGamepad != nil || controller.gamepad != nil;
 }
 
+#pragma clang diagnostic pop
+
 +(int) getGamepadCount {
     int count = 0;
     
@@ -432,38 +596,26 @@
         mask = 0x1;
     }
     
-#if TARGET_OS_IPHONE
     DataManager* dataMan = [[DataManager alloc] init];
-    OnScreenControlsLevel level = (OnScreenControlsLevel)[[dataMan getSettings].onscreenControls integerValue];
+    TemporarySettings* settings = [dataMan getSettings];
+    OnScreenControlsLevel level = (OnScreenControlsLevel)[settings.onscreenControls integerValue];
     
-    // Even if no gamepads are present, we will always count one
-    // if OSC is enabled.
-    if (level != OnScreenControlsLevelOff) {
+    // Even if no gamepads are present, we will always count one if OSC is enabled,
+    // or it's set to auto and no keyboard or mouse is present. Absolute touch mode
+    // disables the OSC.
+    if (level != OnScreenControlsLevelOff && (![ControllerSupport hasKeyboardOrMouse] || level != OnScreenControlsLevelAuto) && !settings.absoluteTouchMode) {
         mask |= 0x1;
     }
-#endif
     
     return mask;
 }
 
--(void) rumbleTimer
+-(NSUInteger) getConnectedGamepadCount
 {
-    for (int i = 0; i < 4; i++) {
-        Controller* controller = [_controllers objectForKey:[NSNumber numberWithInteger:i]];
-        if (controller == nil && i == 0 && _oscEnabled) {
-            // No physical controller, but we have on-screen controls
-            controller = _player0osc;
-        }
-        if (controller == nil) {
-            // No connected controller for this player
-            continue;
-        }
-        
-        [self rumbleController:controller];
-    }
+    return _controllers.count;
 }
 
--(id) initWithConfig:(StreamConfiguration*)streamConfig
+-(id) initWithConfig:(StreamConfiguration*)streamConfig presenceDelegate:(id<InputPresenceDelegate>)delegate
 {
     self = [super init];
     
@@ -471,24 +623,13 @@
     _controllers = [[NSMutableDictionary alloc] init];
     _controllerNumbers = 0;
     _multiController = streamConfig.multiController;
+    _presenceDelegate = delegate;
 
     _player0osc = [[Controller alloc] init];
     _player0osc.playerIndex = 0;
 
-#if TARGET_OS_IPHONE
     DataManager* dataMan = [[DataManager alloc] init];
     _oscEnabled = (OnScreenControlsLevel)[[dataMan getSettings].onscreenControls integerValue] != OnScreenControlsLevelOff;
-#else
-    _oscEnabled = false;
-    initGamepad(self);
-    Gamepad_detectDevices();
-#endif
-    
-    _rumbleTimer = [NSTimer scheduledTimerWithTimeInterval:0.20
-                                                    target:self
-                                                  selector:@selector(rumbleTimer)
-                                                  userInfo:nil
-                                                   repeats:YES];
     
     Log(LOG_I, @"Number of supported controllers connected: %d", [ControllerSupport getGamepadCount]);
     Log(LOG_I, @"Multi-controller: %d", _multiController);
@@ -497,11 +638,16 @@
         if ([ControllerSupport isSupportedGamepad:controller]) {
             [self assignController:controller];
             [self registerControllerCallbacks:controller];
-            [self updateAutoOnScreenControlMode];
         }
     }
     
-    self.connectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCControllerDidConnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        for (GCMouse* mouse in [GCMouse mice]) {
+            [self registerMouseCallbacks:mouse];
+        }
+    }
+    
+    _controllerConnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCControllerDidConnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
         Log(LOG_I, @"Controller connected!");
         
         GCController* controller = note.object;
@@ -518,8 +664,11 @@
         
         // Re-evaluate the on-screen control mode
         [self updateAutoOnScreenControlMode];
+        
+        // Notify the delegate
+        [self->_presenceDelegate gamepadPresenceChanged];
     }];
-    self.disconnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCControllerDidDisconnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+    _controllerDisconnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCControllerDidDisconnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
         Log(LOG_I, @"Controller disconnected!");
         
         GCController* controller = note.object;
@@ -535,6 +684,10 @@
         
         // Unset the GCController on this object (in case it is the OSC, which will persist)
         Controller* limeController = [self->_controllers objectForKey:[NSNumber numberWithInteger:controller.playerIndex]];
+        
+        // Stop haptics on this controller
+        [self cleanupControllerHaptics:limeController];
+        
         limeController.gamepad = nil;
         
         // Inform the server of the updated active gamepads before removing this controller
@@ -543,23 +696,91 @@
 
         // Re-evaluate the on-screen control mode
         [self updateAutoOnScreenControlMode];
+        
+        // Notify the delegate
+        [self->_presenceDelegate gamepadPresenceChanged];
     }];
+    
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        _mouseConnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCMouseDidConnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+            Log(LOG_I, @"Mouse connected!");
+            
+            GCMouse* mouse = note.object;
+            
+            // Register for mouse events
+            [self registerMouseCallbacks: mouse];
+
+            // Re-evaluate the on-screen control mode
+            [self updateAutoOnScreenControlMode];
+            
+            // Notify the delegate
+            [self->_presenceDelegate mousePresenceChanged];
+        }];
+        _mouseDisconnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCMouseDidDisconnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+            Log(LOG_I, @"Mouse disconnected!");
+            
+            GCMouse* mouse = note.object;
+            
+            // Unregister for mouse events
+            [self unregisterMouseCallbacks: mouse];
+
+            // Re-evaluate the on-screen control mode
+            [self updateAutoOnScreenControlMode];
+            
+            // Notify the delegate
+            [self->_presenceDelegate mousePresenceChanged];
+        }];
+        _keyboardConnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCKeyboardDidConnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+            Log(LOG_I, @"Keyboard connected!");
+            
+            // Re-evaluate the on-screen control mode
+            [self updateAutoOnScreenControlMode];
+        }];
+        _keyboardDisconnectObserver = [[NSNotificationCenter defaultCenter] addObserverForName:GCKeyboardDidDisconnectNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification *note) {
+            Log(LOG_I, @"Keyboard disconnected!");
+
+            // Re-evaluate the on-screen control mode
+            [self updateAutoOnScreenControlMode];
+        }];
+    }
+    
     return self;
 }
 
 -(void) cleanup
 {
-    [_rumbleTimer invalidate];
-    _rumbleTimer = nil;
-    [[NSNotificationCenter defaultCenter] removeObserver:self.connectObserver];
-    [[NSNotificationCenter defaultCenter] removeObserver:self.disconnectObserver];
-    [_controllers removeAllObjects];
+    [[NSNotificationCenter defaultCenter] removeObserver:_controllerConnectObserver];
+    [[NSNotificationCenter defaultCenter] removeObserver:_controllerDisconnectObserver];
+    [[NSNotificationCenter defaultCenter] removeObserver:_mouseConnectObserver];
+    [[NSNotificationCenter defaultCenter] removeObserver:_mouseDisconnectObserver];
+    [[NSNotificationCenter defaultCenter] removeObserver:_keyboardConnectObserver];
+    [[NSNotificationCenter defaultCenter] removeObserver:_keyboardDisconnectObserver];
+    
+    _controllerConnectObserver = nil;
+    _controllerDisconnectObserver = nil;
+    _mouseConnectObserver = nil;
+    _mouseDisconnectObserver = nil;
+    _keyboardConnectObserver = nil;
+    _keyboardDisconnectObserver = nil;
+    
     _controllerNumbers = 0;
+    
+    for (Controller* controller in [_controllers allValues]) {
+        [self cleanupControllerHaptics:controller];
+    }
+    [_controllers removeAllObjects];
+    
     for (GCController* controller in [GCController controllers]) {
         if ([ControllerSupport isSupportedGamepad:controller]) {
             [self unregisterControllerCallbacks:controller];
         }
     }
+    
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        for (GCMouse* mouse in [GCMouse mice]) {
+            [self unregisterMouseCallbacks:mouse];
+        }
+    }
 }
 
 @end

Limelight/Input/HapticContext.h

@@ -0,0 +1,20 @@
+//
+//  HapticContext.h
+//  Moonlight
+//
+//  Created by Cameron Gutman on 9/17/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+@import CoreHaptics;
+@import GameController;
+
+@interface HapticContext : NSObject
+
+-(void)setMotorAmplitude:(unsigned short)amplitude;
+-(void)cleanup;
+
++(HapticContext*) createContextForHighFreqMotor:(GCController*)gamepad;
++(HapticContext*) createContextForLowFreqMotor:(GCController*)gamepad;
+
+@end

Limelight/Input/HapticContext.m

@@ -0,0 +1,147 @@
+//
+//  HapticContext.m
+//  Moonlight
+//
+//  Created by Cameron Gutman on 9/17/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+#import "HapticContext.h"
+
+@import CoreHaptics;
+@import GameController;
+
+@implementation HapticContext {
+    GCControllerPlayerIndex _playerIndex;
+    CHHapticEngine* _hapticEngine API_AVAILABLE(ios(13.0), tvos(14.0));
+    id<CHHapticPatternPlayer> _hapticPlayer API_AVAILABLE(ios(13.0), tvos(14.0));
+    BOOL _playing;
+}
+
+-(void)cleanup API_AVAILABLE(ios(14.0), tvos(14.0)) {
+    if (_hapticPlayer != nil) {
+        [_hapticPlayer cancelAndReturnError:nil];
+        _hapticPlayer = nil;
+    }
+    if (_hapticEngine != nil) {
+        [_hapticEngine stopWithCompletionHandler:nil];
+        _hapticEngine = nil;
+    }
+}
+
+-(void)setMotorAmplitude:(unsigned short)amplitude API_AVAILABLE(ios(14.0), tvos(14.0)) {
+    NSError* error;
+
+    // Check if the haptic engine died
+    if (_hapticEngine == nil) {
+        return;
+    }
+    
+    // Stop the effect entirely if the amplitude is 0
+    if (amplitude == 0) {
+        if (_playing) {
+            [_hapticPlayer stopAtTime:0 error:&error];
+            _playing = NO;
+        }
+        
+        return;
+    }
+
+    if (_hapticPlayer == nil) {
+        // We must initialize the intensity to 1.0f because the dynamic parameters are multiplied by this value before being applied
+        CHHapticEventParameter* intensityParameter = [[CHHapticEventParameter alloc] initWithParameterID:CHHapticEventParameterIDHapticIntensity value:1.0f];
+        CHHapticEvent* hapticEvent = [[CHHapticEvent alloc] initWithEventType:CHHapticEventTypeHapticContinuous parameters:[NSArray arrayWithObject:intensityParameter] relativeTime:0 duration:GCHapticDurationInfinite];
+        CHHapticPattern* hapticPattern = [[CHHapticPattern alloc] initWithEvents:[NSArray arrayWithObject:hapticEvent] parameters:[[NSArray alloc] init] error:&error];
+        if (error != nil) {
+            Log(LOG_W, @"Controller %d: Haptic pattern creation failed: %@", _playerIndex, error);
+            return;
+        }
+        
+        _hapticPlayer = [_hapticEngine createPlayerWithPattern:hapticPattern error:&error];
+        if (error != nil) {
+            Log(LOG_W, @"Controller %d: Haptic player creation failed: %@", _playerIndex, error);
+            return;
+        }
+    }
+
+    CHHapticDynamicParameter* intensityParameter = [[CHHapticDynamicParameter alloc] initWithParameterID:CHHapticDynamicParameterIDHapticIntensityControl value:amplitude / 65535.0f relativeTime:0];
+    [_hapticPlayer sendParameters:[NSArray arrayWithObject:intensityParameter] atTime:CHHapticTimeImmediate error:&error];
+    if (error != nil) {
+        Log(LOG_W, @"Controller %d: Haptic player parameter update failed: %@", _playerIndex, error);
+        return;
+    }
+    
+    if (!_playing) {
+        [_hapticPlayer startAtTime:0 error:&error];
+        if (error != nil) {
+            _hapticPlayer = nil;
+            Log(LOG_W, @"Controller %d: Haptic playback start failed: %@", _playerIndex, error);
+            return;
+        }
+        
+        _playing = YES;
+    }
+}
+
+-(id) initWithGamepad:(GCController*)gamepad locality:(GCHapticsLocality)locality API_AVAILABLE(ios(14.0), tvos(14.0)) {
+    if (gamepad.haptics == nil) {
+        Log(LOG_W, @"Controller %d does not support haptics", gamepad.playerIndex);
+        return nil;
+    }
+    
+    _playerIndex = gamepad.playerIndex;
+    _hapticEngine = [gamepad.haptics createEngineWithLocality:locality];
+    
+    NSError* error;
+    [_hapticEngine startAndReturnError:&error];
+    if (error != nil) {
+        Log(LOG_W, @"Controller %d: Haptic engine failed to start: %@", gamepad.playerIndex, error);
+        return nil;
+    }
+    
+    __weak typeof(self) weakSelf = self;
+    _hapticEngine.stoppedHandler = ^(CHHapticEngineStoppedReason stoppedReason) {
+        HapticContext* me = weakSelf;
+        if (me == nil) {
+            return;
+        }
+        
+        Log(LOG_W, @"Controller %d: Haptic engine stopped: %p", me->_playerIndex, stoppedReason);
+        me->_hapticPlayer = nil;
+        me->_hapticEngine = nil;
+        me->_playing = NO;
+    };
+    _hapticEngine.resetHandler = ^{
+        HapticContext* me = weakSelf;
+        if (me == nil) {
+            return;
+        }
+        
+        Log(LOG_W, @"Controller %d: Haptic engine reset", me->_playerIndex);
+        me->_hapticPlayer = nil;
+        me->_playing = NO;
+        [me->_hapticEngine startAndReturnError:nil];
+    };
+    
+    return self;
+}
+
++(HapticContext*) createContextForHighFreqMotor:(GCController*)gamepad {
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        return [[HapticContext alloc] initWithGamepad:gamepad locality:GCHapticsLocalityRightHandle];
+    }
+    else {
+        return nil;
+    }
+}
+
++(HapticContext*) createContextForLowFreqMotor:(GCController*)gamepad {
+    if (@available(iOS 14.0, tvOS 14.0, *)) {
+        return [[HapticContext alloc] initWithGamepad:gamepad locality:GCHapticsLocalityLeftHandle];
+    }
+    else {
+        return nil;
+    }
+}
+
+@end

Limelight/Input/KeyboardSupport.h

@@ -16,6 +16,7 @@ struct KeyEvent {
     u_char modifier;
 };
 
++ (BOOL)sendKeyEvent:(UIKey*)key down:(BOOL)down API_AVAILABLE(ios(13.4));
 + (struct KeyEvent) translateKeyEvent:(unichar) inputChar withModifierFlags:(UIKeyModifierFlags)modifierFlags;
 
 @end

Limelight/Input/KeyboardSupport.m

@@ -11,6 +11,223 @@
 
 @implementation KeyboardSupport
 
++ (BOOL)sendKeyEvent:(UIKey*)key down:(BOOL)down API_AVAILABLE(ios(13.4)){
+    char modifierFlags = 0;
+    short keyCode = 0;
+    
+    if (key.modifierFlags & UIKeyModifierShift) {
+        modifierFlags |= MODIFIER_SHIFT;
+    }
+    if (key.modifierFlags & UIKeyModifierAlternate) {
+        modifierFlags |= MODIFIER_ALT;
+    }
+    if (key.modifierFlags & UIKeyModifierControl) {
+        modifierFlags |= MODIFIER_CTRL;
+    }
+    if (key.modifierFlags & UIKeyModifierCommand) {
+        modifierFlags |= MODIFIER_META;
+    }
+    
+    // This converts UIKeyboardHIDUsage values to Win32 VK_* values
+    // https://docs.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes
+    if (key.keyCode >= UIKeyboardHIDUsageKeyboardA &&
+        key.keyCode <= UIKeyboardHIDUsageKeyboardZ) {
+        keyCode = (key.keyCode - UIKeyboardHIDUsageKeyboardA) + 0x41;
+    }
+    else if (key.keyCode == UIKeyboardHIDUsageKeyboard0) {
+        // This key is at the beginning of the VK_ range but the end
+        // of the UIKeyboardHIDUsageKeyboard range.
+        keyCode = 0x30;
+    }
+    else if (key.keyCode >= UIKeyboardHIDUsageKeyboard1 &&
+             key.keyCode <= UIKeyboardHIDUsageKeyboard9) {
+        keyCode = (key.keyCode - UIKeyboardHIDUsageKeyboard1) + 0x31;
+    }
+    else if (key.keyCode == UIKeyboardHIDUsageKeypad0) {
+        // This key is at the beginning of the VK_ range but the end
+        // of the UIKeyboardHIDUsageKeypad range.
+        keyCode = 0x60;
+    }
+    else if (key.keyCode >= UIKeyboardHIDUsageKeypad1 &&
+             key.keyCode <= UIKeyboardHIDUsageKeypad9) {
+        keyCode = (key.keyCode - UIKeyboardHIDUsageKeypad1) + 0x61;
+    }
+    else if (key.keyCode >= UIKeyboardHIDUsageKeyboardF1 &&
+             key.keyCode <= UIKeyboardHIDUsageKeyboardF12) {
+        keyCode = (key.keyCode - UIKeyboardHIDUsageKeyboardF1) + 0x70;
+    }
+    else if (key.keyCode >= UIKeyboardHIDUsageKeyboardF13 &&
+             key.keyCode <= UIKeyboardHIDUsageKeyboardF24) {
+        keyCode = (key.keyCode - UIKeyboardHIDUsageKeyboardF13) + 0x7C;
+    }
+    else {
+        switch (key.keyCode) {
+            case UIKeyboardHIDUsageKeyboardReturnOrEnter:
+                keyCode = 0x0D;
+                break;
+            case UIKeyboardHIDUsageKeyboardEscape:
+                keyCode = 0x1B;
+                break;
+            case UIKeyboardHIDUsageKeyboardDeleteOrBackspace:
+                keyCode = 0x08;
+                break;
+            case UIKeyboardHIDUsageKeyboardTab:
+                keyCode = 0x09;
+                break;
+            case UIKeyboardHIDUsageKeyboardSpacebar:
+                keyCode = 0x20;
+                break;
+            case UIKeyboardHIDUsageKeyboardHyphen:
+                keyCode = 0xBD;
+                break;
+            case UIKeyboardHIDUsageKeyboardEqualSign:
+                keyCode = 0xBB;
+                break;
+            case UIKeyboardHIDUsageKeyboardOpenBracket:
+                keyCode = 0xDB;
+                break;
+            case UIKeyboardHIDUsageKeyboardCloseBracket:
+                keyCode = 0xDD;
+                break;
+            case UIKeyboardHIDUsageKeyboardBackslash:
+                keyCode = 0xDC;
+                break;
+            case UIKeyboardHIDUsageKeyboardSemicolon:
+                keyCode = 0xBA;
+                break;
+            case UIKeyboardHIDUsageKeyboardQuote:
+                keyCode = 0xDE;
+                break;
+            case UIKeyboardHIDUsageKeyboardGraveAccentAndTilde:
+                keyCode = 0xC0;
+                break;
+            case UIKeyboardHIDUsageKeyboardComma:
+                keyCode = 0xBC;
+                break;
+            case UIKeyboardHIDUsageKeyboardPeriod:
+                keyCode = 0xBE;
+                break;
+            case UIKeyboardHIDUsageKeyboardSlash:
+                keyCode = 0xBF;
+                break;
+            case UIKeyboardHIDUsageKeyboardCapsLock:
+                keyCode = 0x14;
+                break;
+            case UIKeyboardHIDUsageKeyboardPrintScreen:
+                keyCode = 0x2A;
+                break;
+            case UIKeyboardHIDUsageKeyboardScrollLock:
+                keyCode = 0x91;
+                break;
+            case UIKeyboardHIDUsageKeyboardPause:
+                keyCode = 0x13;
+                break;
+            case UIKeyboardHIDUsageKeyboardInsert:
+                keyCode = 0x2D;
+                break;
+            case UIKeyboardHIDUsageKeyboardHome:
+                keyCode = 0x24;
+                break;
+            case UIKeyboardHIDUsageKeyboardPageUp:
+                keyCode = 0x21;
+                break;
+            case UIKeyboardHIDUsageKeyboardDeleteForward:
+                keyCode = 0x2E;
+                break;
+            case UIKeyboardHIDUsageKeyboardEnd:
+                keyCode = 0x23;
+                break;
+            case UIKeyboardHIDUsageKeyboardPageDown:
+                keyCode = 0x22;
+                break;
+            case UIKeyboardHIDUsageKeyboardRightArrow:
+                keyCode = 0x27;
+                break;
+            case UIKeyboardHIDUsageKeyboardLeftArrow:
+                keyCode = 0x25;
+                break;
+            case UIKeyboardHIDUsageKeyboardDownArrow:
+                keyCode = 0x28;
+                break;
+            case UIKeyboardHIDUsageKeyboardUpArrow:
+                keyCode = 0x26;
+                break;
+            case UIKeyboardHIDUsageKeypadNumLock:
+                keyCode = 0x90;
+                break;
+            case UIKeyboardHIDUsageKeypadSlash:
+                keyCode = 0x6F;
+                break;
+            case UIKeyboardHIDUsageKeypadAsterisk:
+                keyCode = 0x6A;
+                break;
+            case UIKeyboardHIDUsageKeypadHyphen:
+                keyCode = 0x6D;
+                break;
+            case UIKeyboardHIDUsageKeypadPlus:
+                keyCode = 0x6B;
+                break;
+            case UIKeyboardHIDUsageKeypadEnter:
+                keyCode = 0x0D; // FIXME: Is this correct?
+                break;
+            case UIKeyboardHIDUsageKeypadPeriod:
+                keyCode = 0x6E;
+                break;
+            case UIKeyboardHIDUsageKeyboardNonUSBackslash:
+                keyCode = 0xE2;
+                break;
+            case UIKeyboardHIDUsageKeypadComma:
+                keyCode = 0x6C;
+                break;
+            case UIKeyboardHIDUsageKeyboardCancel:
+                keyCode = 0x03;
+                break;
+            case UIKeyboardHIDUsageKeyboardClear:
+                keyCode = 0x0C;
+                break;
+            case UIKeyboardHIDUsageKeyboardCrSelOrProps:
+                keyCode = 0xF7;
+                break;
+            case UIKeyboardHIDUsageKeyboardExSel:
+                keyCode = 0xF8;
+                break;
+            case UIKeyboardHIDUsageKeyboardLeftGUI:
+                keyCode = 0x5B;
+                break;
+            case UIKeyboardHIDUsageKeyboardLeftControl:
+                keyCode = 0xA2;
+                break;
+            case UIKeyboardHIDUsageKeyboardLeftShift:
+                keyCode = 0xA0;
+                break;
+            case UIKeyboardHIDUsageKeyboardLeftAlt:
+                keyCode = 0xA4;
+                break;
+            case UIKeyboardHIDUsageKeyboardRightGUI:
+                keyCode = 0x5C;
+                break;
+            case UIKeyboardHIDUsageKeyboardRightControl:
+                keyCode = 0xA3;
+                break;
+            case UIKeyboardHIDUsageKeyboardRightShift:
+                keyCode = 0xA1;
+                break;
+            case UIKeyboardHIDUsageKeyboardRightAlt:
+                keyCode = 0xA5;
+                break;
+            default:
+                NSLog(@"Unhandled HID usage: %lu", (unsigned long)key.keyCode);
+                assert(0);
+                return false;
+        }
+    }
+    
+    LiSendKeyboardEvent(0x8000 | keyCode,
+                        down ? KEY_ACTION_DOWN : KEY_ACTION_UP,
+                        modifierFlags);
+    return true;
+}
+
 + (struct KeyEvent)translateKeyEvent:(unichar)inputChar withModifierFlags:(UIKeyModifierFlags)modifierFlags {
     struct KeyEvent event;
     event.keycode = 0;
@@ -25,10 +242,12 @@
         case UIKeyModifierControl:
             [KeyboardSupport addControlModifier:&event];
             break;
+        case UIKeyModifierCommand:
+            [KeyboardSupport addMetaModifier:&event];
+            break;
         case UIKeyModifierAlternate:
             [KeyboardSupport addAltModifier:&event];
             break;
-        case UIKeyModifierCommand:
         case UIKeyModifierNumericPad:
             break;
     }
@@ -183,6 +402,11 @@
     event->modifierKeycode = 0x11;
 }
 
++ (void) addMetaModifier:(struct KeyEvent*)event {
+    event->modifier = MODIFIER_META;
+    event->modifierKeycode = 0x5B;
+}
+
 + (void) addAltModifier:(struct KeyEvent*)event {
     event->modifier = MODIFIER_ALT;
     event->modifierKeycode = 0x12;

Limelight/Input/OnScreenControls.h

@@ -7,7 +7,6 @@
 //
 
 #import <Foundation/Foundation.h>
-#import "StreamView.h"
 
 @class ControllerSupport;
 
@@ -25,12 +24,12 @@ typedef NS_ENUM(NSInteger, OnScreenControlsLevel) {
     OnScreenControlsLevelAutoGCExtendedGamepadWithStickButtons
 };
 
-#if TARGET_OS_IPHONE
-- (id) initWithView:(UIView*)view controllerSup:(ControllerSupport*)controllerSupport swipeDelegate:(id<EdgeDetectionDelegate>)edgeDelegate;
+- (id) initWithView:(UIView*)view controllerSup:(ControllerSupport*)controllerSupport;
 - (BOOL) handleTouchDownEvent:(NSSet*)touches;
 - (BOOL) handleTouchUpEvent:(NSSet*)touches;
 - (BOOL) handleTouchMovedEvent:(NSSet*)touches;
 - (void) setLevel:(OnScreenControlsLevel)level;
-#endif
+- (OnScreenControlsLevel) getLevel;
+- (void) show;
 
 @end

Limelight/Input/OnScreenControls.m

@@ -7,6 +7,7 @@
 //
 
 #import "OnScreenControls.h"
+#import "StreamView.h"
 #import "ControllerSupport.h"
 #import "Controller.h"
 #include "Limelight.h"
@@ -35,7 +36,6 @@
     CALayer* _l1Button;
     CALayer* _l2Button;
     CALayer* _l3Button;
-    CALayer* _edge;
     
     UITouch* _aTouch;
     UITouch* _bTouch;
@@ -52,7 +52,6 @@
     UITouch* _l1Touch;
     UITouch* _l2Touch;
     UITouch* _l3Touch;
-    UITouch* _edgeTouch;
     
     NSDate* l3TouchStart;
     NSDate* r3TouchStart;
@@ -64,10 +63,10 @@
     CGRect _controlArea;
     UIView* _view;
     OnScreenControlsLevel _level;
+    BOOL _visible;
     
     ControllerSupport *_controllerSupport;
     Controller *_controller;
-    id<EdgeDetectionDelegate> _edgeDelegate;
     NSMutableArray* _deadTouches;
 }
 
@@ -112,15 +111,14 @@ static float L2_Y;
 static float L3_X;
 static float L3_Y;
 
-- (id) initWithView:(UIView*)view controllerSup:(ControllerSupport*)controllerSupport swipeDelegate:(id<EdgeDetectionDelegate>)swipeDelegate {
+- (id) initWithView:(UIView*)view controllerSup:(ControllerSupport*)controllerSupport {
     self = [self init];
     _view = view;
     _controllerSupport = controllerSupport;
     _controller = [controllerSupport getOscController];
-    _edgeDelegate = swipeDelegate;
     _deadTouches = [[NSMutableArray alloc] init];
     
-    _iPad = (UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPad);
+    _iPad = ([UIDevice currentDevice].userInterfaceIdiom == UIUserInterfaceIdiomPad);
     _controlArea = CGRectMake(0, 0, _view.frame.size.width, _view.frame.size.height);
     if (_iPad)
     {
@@ -154,16 +152,28 @@ static float L3_Y;
     _rightStickBackground = [CALayer layer];
     _leftStick = [CALayer layer];
     _rightStick = [CALayer layer];
-    _edge = [CALayer layer];
-    
-    [self setupEdgeDetection];
     
     return self;
 }
 
+- (void) show {
+    _visible = YES;
+        
+    [self updateControls];
+}
+
 - (void) setLevel:(OnScreenControlsLevel)level {
     _level = level;
-    [self updateControls];
+    
+    // Only update controls if we're showing, otherwise
+    // show will do it for us.
+    if (_visible) {
+        [self updateControls];
+    }
+}
+
+- (OnScreenControlsLevel) getLevel {
+    return _level;
 }
 
 - (void) updateControls {
@@ -237,11 +247,6 @@ static float L3_Y;
     }
 }
 
-- (void) setupEdgeDetection {
-    _edge.frame = CGRectMake(0, 0, _view.frame.size.width * EDGE_WIDTH, _view.frame.size.height);
-    [_view.layer addSublayer:_edge];
-}
-
 // For GCExtendedGamepad controls we move start, select, L3, and R3 to the button
 - (void) setupExtendedGamepadControls {
     // Start with the default complex layout
@@ -641,7 +646,7 @@ static float L3_Y;
     if (updated) {
         [_controllerSupport updateFinished:_controller];
     }
-        return updated || buttonTouch;
+    return updated || buttonTouch;
 }
 
 - (BOOL)handleTouchDownEvent:touches {
@@ -752,8 +757,6 @@ static float L3_Y;
             }
             _rsTouch = touch;
             stickTouch = true;
-        } else if ([_edge.presentationLayer hitTest:touchLocation]) {
-            _edgeTouch = touch;
         }
         if (!updated && !stickTouch && [self isInDeadZone:touch]) {
             [_deadTouches addObject:touch];
@@ -837,11 +840,6 @@ static float L3_Y;
         else if (touch == _r3Touch) {
             _r3Touch = nil;
             touched = true;
-        } else if (touch == _edgeTouch) {
-            _edgeTouch = nil;
-            if ([touch locationInView:_view].x >= _view.frame.size.width / 4) {
-                [_edgeDelegate edgeSwiped];
-            }
         }
         if ([_deadTouches containsObject:touch]) {
             [_deadTouches removeObject:touch];

Limelight/Input/RelativeTouchHandler.h

@@ -0,0 +1,19 @@
+//
+//  RelativeTouchHandler.h
+//  Moonlight
+//
+//  Created by Cameron Gutman on 11/1/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+#import "StreamView.h"
+
+NS_ASSUME_NONNULL_BEGIN
+
+@interface RelativeTouchHandler : UIResponder
+
+-(id)initWithView:(StreamView*)view;
+
+@end
+
+NS_ASSUME_NONNULL_END

Limelight/Input/RelativeTouchHandler.m

@@ -0,0 +1,204 @@
+//
+//  RelativeTouchHandler.m
+//  Moonlight
+//
+//  Created by Cameron Gutman on 11/1/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+#import "RelativeTouchHandler.h"
+
+#include <Limelight.h>
+
+static const int REFERENCE_WIDTH = 1280;
+static const int REFERENCE_HEIGHT = 720;
+
+@implementation RelativeTouchHandler {
+    CGPoint touchLocation, originalLocation;
+    BOOL touchMoved;
+    BOOL isDragging;
+    NSTimer* dragTimer;
+    
+#if TARGET_OS_TV
+    UIGestureRecognizer* remotePressRecognizer;
+    UIGestureRecognizer* remoteLongPressRecognizer;
+#endif
+    
+    UIView* view;
+}
+
+- (id)initWithView:(StreamView*)view {
+    self = [self init];
+    self->view = view;
+    
+#if TARGET_OS_TV
+    remotePressRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(remoteButtonPressed:)];
+    remotePressRecognizer.allowedPressTypes = @[@(UIPressTypeSelect)];
+    
+    remoteLongPressRecognizer = [[UILongPressGestureRecognizer alloc] initWithTarget:self action:@selector(remoteButtonLongPressed:)];
+    remoteLongPressRecognizer.allowedPressTypes = @[@(UIPressTypeSelect)];
+    
+    [self->view addGestureRecognizer:remotePressRecognizer];
+    [self->view addGestureRecognizer:remoteLongPressRecognizer];
+#endif
+    
+    return self;
+}
+
+- (BOOL)isConfirmedMove:(CGPoint)currentPoint from:(CGPoint)originalPoint {
+    // Movements of greater than 5 pixels are considered confirmed
+    return hypotf(originalPoint.x - currentPoint.x, originalPoint.y - currentPoint.y) >= 5;
+}
+
+- (void)onDragStart:(NSTimer*)timer {
+    if (!touchMoved && !isDragging){
+        isDragging = true;
+        LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+    }
+}
+
+- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
+    touchMoved = false;
+    if ([[event allTouches] count] == 1) {
+        UITouch *touch = [[event allTouches] anyObject];
+        originalLocation = touchLocation = [touch locationInView:view];
+        if (!isDragging) {
+            dragTimer = [NSTimer scheduledTimerWithTimeInterval:0.650
+                                                     target:self
+                                                   selector:@selector(onDragStart:)
+                                                   userInfo:nil
+                                                    repeats:NO];
+        }
+    }
+    else if ([[event allTouches] count] == 2) {
+        CGPoint firstLocation = [[[[event allTouches] allObjects] objectAtIndex:0] locationInView:view];
+        CGPoint secondLocation = [[[[event allTouches] allObjects] objectAtIndex:1] locationInView:view];
+        
+        originalLocation = touchLocation = CGPointMake((firstLocation.x + secondLocation.x) / 2, (firstLocation.y + secondLocation.y) / 2);
+    }
+}
+
+- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
+    if ([[event allTouches] count] == 1) {
+        UITouch *touch = [[event allTouches] anyObject];
+        CGPoint currentLocation = [touch locationInView:view];
+        
+        if (touchLocation.x != currentLocation.x ||
+            touchLocation.y != currentLocation.y)
+        {
+            int deltaX = (currentLocation.x - touchLocation.x) * (REFERENCE_WIDTH / view.bounds.size.width);
+            int deltaY = (currentLocation.y - touchLocation.y) * (REFERENCE_HEIGHT / view.bounds.size.height);
+            
+            if (deltaX != 0 || deltaY != 0) {
+                LiSendMouseMoveEvent(deltaX, deltaY);
+                touchLocation = currentLocation;
+                
+                // If we've moved far enough to confirm this wasn't just human/machine error,
+                // mark it as such.
+                if ([self isConfirmedMove:touchLocation from:originalLocation]) {
+                    touchMoved = true;
+                }
+            }
+        }
+    } else if ([[event allTouches] count] == 2) {
+        CGPoint firstLocation = [[[[event allTouches] allObjects] objectAtIndex:0] locationInView:view];
+        CGPoint secondLocation = [[[[event allTouches] allObjects] objectAtIndex:1] locationInView:view];
+        
+        CGPoint avgLocation = CGPointMake((firstLocation.x + secondLocation.x) / 2, (firstLocation.y + secondLocation.y) / 2);
+        if (touchLocation.y != avgLocation.y) {
+            LiSendHighResScrollEvent((avgLocation.y - touchLocation.y) * 10);
+        }
+
+        // If we've moved far enough to confirm this wasn't just human/machine error,
+        // mark it as such.
+        if ([self isConfirmedMove:firstLocation from:originalLocation]) {
+            touchMoved = true;
+        }
+        
+        touchLocation = avgLocation;
+    }
+}
+
+- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
+    [dragTimer invalidate];
+    dragTimer = nil;
+    if (isDragging) {
+        isDragging = false;
+        LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+    } else if (!touchMoved) {
+        if ([[event allTouches] count]  == 2) {
+            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
+                Log(LOG_D, @"Sending right mouse button press");
+                
+                LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_RIGHT);
+                
+                // Wait 100 ms to simulate a real button press
+                usleep(100 * 1000);
+                
+                LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_RIGHT);
+            });
+        } else if ([[event allTouches] count]  == 1) {
+            dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
+                if (!self->isDragging){
+                    Log(LOG_D, @"Sending left mouse button press");
+                    
+                    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+                    
+                    // Wait 100 ms to simulate a real button press
+                    usleep(100 * 1000);
+                }
+                self->isDragging = false;
+                LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+            });
+        }
+    }
+    
+    // We we're moving from 2+ touches to 1. Synchronize the current position
+    // of the active finger so we don't jump unexpectedly on the next touchesMoved
+    // callback when finger 1 switches on us.
+    if ([[event allTouches] count] - [touches count] == 1) {
+        NSMutableSet *activeSet = [[NSMutableSet alloc] initWithCapacity:[[event allTouches] count]];
+        [activeSet unionSet:[event allTouches]];
+        [activeSet minusSet:touches];
+        touchLocation = [[activeSet anyObject] locationInView:view];
+        
+        // Mark this touch as moved so we don't send a left mouse click if the user
+        // right clicks without moving their other finger.
+        touchMoved = true;
+    }
+}
+
+- (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
+    [dragTimer invalidate];
+    dragTimer = nil;
+    if (isDragging) {
+        isDragging = false;
+        LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+    }
+}
+
+#if TARGET_OS_TV
+- (void)remoteButtonPressed:(id)sender {
+    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
+        Log(LOG_D, @"Sending left mouse button press");
+        
+        // Mark this as touchMoved to avoid a duplicate press on touch up
+        self->touchMoved = true;
+        
+        LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+        
+        // Wait 100 ms to simulate a real button press
+        usleep(100 * 1000);
+            
+        LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
+    });
+}
+- (void)remoteButtonLongPressed:(id)sender {
+    Log(LOG_D, @"Holding left mouse button");
+    
+    isDragging = true;
+    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+}
+#endif
+
+@end

Limelight/Input/StreamView.h

@@ -7,18 +7,31 @@
 //
 
 #import "ControllerSupport.h"
+#import "OnScreenControls.h"
+#import "Moonlight-Swift.h"
+#import "StreamConfiguration.h"
 
-@protocol EdgeDetectionDelegate <NSObject>
+@protocol UserInteractionDelegate <NSObject>
 
-- (void) edgeSwiped;
+- (void) userInteractionBegan;
+- (void) userInteractionEnded;
 
 @end
 
-@interface StreamView : OSView <UITextFieldDelegate>
+#if TARGET_OS_TV
+@interface StreamView : UIView <X1KitMouseDelegate, UITextFieldDelegate>
+#else
+@interface StreamView : UIView <X1KitMouseDelegate, UITextFieldDelegate, UIPointerInteractionDelegate>
+#endif
 
-@property (nonatomic, retain) IBOutlet UITextField* keyInputField;
+- (void) setupStreamView:(ControllerSupport*)controllerSupport
+     interactionDelegate:(id<UserInteractionDelegate>)interactionDelegate
+                  config:(StreamConfiguration*)streamConfig;
+- (void) showOnScreenControls;
+- (OnScreenControlsLevel) getCurrentOscState;
 
-- (void) setupOnScreenControls:(ControllerSupport*)controllerSupport swipeDelegate:(id<EdgeDetectionDelegate>)swipeDelegate;
-- (void) setMouseDeltaFactors:(float)x y:(float)y;
+#if !TARGET_OS_TV
+- (void) updateCursorLocation:(CGPoint)location isMouse:(BOOL)isMouse;
+#endif
 
 @end

Limelight/Input/StreamView.m

@@ -8,195 +8,508 @@
 
 #import "StreamView.h"
 #include <Limelight.h>
-#import "OnScreenControls.h"
 #import "DataManager.h"
 #import "ControllerSupport.h"
 #import "KeyboardSupport.h"
+#import "RelativeTouchHandler.h"
+#import "AbsoluteTouchHandler.h"
+
+static const double X1_MOUSE_SPEED_DIVISOR = 2.5;
 
 @implementation StreamView {
-    CGPoint touchLocation, originalLocation;
-    BOOL touchMoved;
     OnScreenControls* onScreenControls;
     
+    UITextField* keyInputField;
     BOOL isInputingText;
-    BOOL isDragging;
-    NSTimer* dragTimer;
     
-    float xDeltaFactor;
-    float yDeltaFactor;
-    float screenFactor;
+    float streamAspectRatio;
+    
+    // iOS 13.4 mouse support
+    NSInteger lastMouseButtonMask;
+    float lastMouseX;
+    float lastMouseY;
+    CGPoint lastScrollTranslation;
+    
+    // Citrix X1 mouse support
+    X1Mouse* x1mouse;
+    double accumulatedMouseDeltaX;
+    double accumulatedMouseDeltaY;
+    
+    UIResponder* touchHandler;
+    
+    id<UserInteractionDelegate> interactionDelegate;
+    NSTimer* interactionTimer;
+    BOOL hasUserInteracted;
     
     NSDictionary<NSString *, NSNumber *> *dictCodes;
 }
 
-- (void) setMouseDeltaFactors:(float)x y:(float)y {
-    xDeltaFactor = x;
-    yDeltaFactor = y;
+- (void) setupStreamView:(ControllerSupport*)controllerSupport
+     interactionDelegate:(id<UserInteractionDelegate>)interactionDelegate
+                  config:(StreamConfiguration*)streamConfig {
+    self->interactionDelegate = interactionDelegate;
+    self->streamAspectRatio = (float)streamConfig.width / (float)streamConfig.height;
     
-    screenFactor = [[UIScreen mainScreen] scale];
-}
-
-- (void) setupOnScreenControls:(ControllerSupport*)controllerSupport swipeDelegate:(id<EdgeDetectionDelegate>)swipeDelegate {
-    onScreenControls = [[OnScreenControls alloc] initWithView:self controllerSup:controllerSupport swipeDelegate:swipeDelegate];
-    DataManager* dataMan = [[DataManager alloc] init];
-    OnScreenControlsLevel level = (OnScreenControlsLevel)[[dataMan getSettings].onscreenControls integerValue];
+    TemporarySettings* settings = [[[DataManager alloc] init] getSettings];
     
-    if (level == OnScreenControlsLevelAuto) {
+    keyInputField = [[UITextField alloc] initWithFrame:CGRectZero];
+    [keyInputField setKeyboardType:UIKeyboardTypeDefault];
+    [keyInputField setAutocorrectionType:UITextAutocorrectionTypeNo];
+    [keyInputField setAutocapitalizationType:UITextAutocapitalizationTypeNone];
+    [keyInputField setSpellCheckingType:UITextSpellCheckingTypeNo];
+    [self addSubview:keyInputField];
+    
+#if TARGET_OS_TV
+    // tvOS requires RelativeTouchHandler to manage Apple Remote input
+    self->touchHandler = [[RelativeTouchHandler alloc] initWithView:self];
+#else
+    // iOS uses RelativeTouchHandler or AbsoluteTouchHandler depending on user preference
+    if (settings.absoluteTouchMode) {
+        self->touchHandler = [[AbsoluteTouchHandler alloc] initWithView:self];
+    }
+    else {
+        self->touchHandler = [[RelativeTouchHandler alloc] initWithView:self];
+    }
+    
+    onScreenControls = [[OnScreenControls alloc] initWithView:self controllerSup:controllerSupport];
+    OnScreenControlsLevel level = (OnScreenControlsLevel)[settings.onscreenControls integerValue];
+    if (settings.absoluteTouchMode) {
+        Log(LOG_I, @"On-screen controls disabled in absolute touch mode");
+        [onScreenControls setLevel:OnScreenControlsLevelOff];
+    }
+    else if (level == OnScreenControlsLevelAuto) {
         [controllerSupport initAutoOnScreenControlMode:onScreenControls];
     }
     else {
         Log(LOG_I, @"Setting manual on-screen controls level: %d", (int)level);
         [onScreenControls setLevel:level];
     }
+    
+    // It would be nice to just use GCMouse on iOS 14+ and the older API on iOS 13
+    // but unfortunately that isn't possible today. GCMouse doesn't recognize many
+    // mice correctly, but UIKit does. We will register for both and ignore UIKit
+    // events if a GCMouse is connected.
+    if (@available(iOS 13.4, *)) {
+        [self addInteraction:[[UIPointerInteraction alloc] initWithDelegate:self]];
+        
+        UIPanGestureRecognizer *discreteMouseWheelRecognizer = [[UIPanGestureRecognizer alloc] initWithTarget:self action:@selector(mouseWheelMovedDiscrete:)];
+        discreteMouseWheelRecognizer.maximumNumberOfTouches = 0;
+        discreteMouseWheelRecognizer.allowedScrollTypesMask = UIScrollTypeMaskDiscrete;
+        discreteMouseWheelRecognizer.allowedTouchTypes = @[@(UITouchTypeIndirectPointer)];
+        [self addGestureRecognizer:discreteMouseWheelRecognizer];
+        
+        UIPanGestureRecognizer *continuousMouseWheelRecognizer = [[UIPanGestureRecognizer alloc] initWithTarget:self action:@selector(mouseWheelMovedContinuous:)];
+        continuousMouseWheelRecognizer.maximumNumberOfTouches = 0;
+        continuousMouseWheelRecognizer.allowedScrollTypesMask = UIScrollTypeMaskContinuous;
+        continuousMouseWheelRecognizer.allowedTouchTypes = @[@(UITouchTypeIndirectPointer)];
+        [self addGestureRecognizer:continuousMouseWheelRecognizer];
+    }
+#endif
+    
+    x1mouse = [[X1Mouse alloc] init];
+    x1mouse.delegate = self;
+    
+    if (settings.btMouseSupport) {
+        [x1mouse start];
+    }
+    
+    // This is critical to ensure keyboard events are delivered to this
+    // StreamView and not our parent UIView, especially on tvOS.
     [self becomeFirstResponder];
 }
 
-- (Boolean)isConfirmedMove:(CGPoint)currentPoint from:(CGPoint)originalPoint {
-    // Movements of greater than 10 pixels are considered confirmed
-    return hypotf(originalPoint.x - currentPoint.x, originalPoint.y - currentPoint.y) >= 10;
+- (void)startInteractionTimer {
+    // Restart user interaction tracking
+    hasUserInteracted = NO;
+    
+    BOOL timerAlreadyRunning = interactionTimer != nil;
+    
+    // Start/restart the timer
+    [interactionTimer invalidate];
+    interactionTimer = [NSTimer scheduledTimerWithTimeInterval:2.0
+                        target:self
+                        selector:@selector(interactionTimerExpired:)
+                        userInfo:nil
+                        repeats:NO];
+    
+    // Notify the delegate if this was a new user interaction
+    if (!timerAlreadyRunning) {
+        [interactionDelegate userInteractionBegan];
+    }
+}
+
+- (void)interactionTimerExpired:(NSTimer *)timer {
+    if (!hasUserInteracted) {
+        // User has finished touching the screen
+        interactionTimer = nil;
+        [interactionDelegate userInteractionEnded];
+    }
+    else {
+        // User is still touching the screen. Restart the timer.
+        [self startInteractionTimer];
+    }
+}
+
+- (void) showOnScreenControls {
+#if !TARGET_OS_TV
+    [onScreenControls show];
+#endif
+}
+
+- (OnScreenControlsLevel) getCurrentOscState {
+    if (onScreenControls == nil) {
+        return OnScreenControlsLevelOff;
+    }
+    else {
+        return [onScreenControls getLevel];
+    }
 }
 
 - (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
+    if ([self handleMouseButtonEvent:BUTTON_ACTION_PRESS
+                          forTouches:touches
+                           withEvent:event]) {
+        // If it's a mouse event, we're done
+        return;
+    }
+    
     Log(LOG_D, @"Touch down");
+    
+    // Notify of user interaction and start expiration timer
+    [self startInteractionTimer];
+    
     if (![onScreenControls handleTouchDownEvent:touches]) {
-        UITouch *touch = [[event allTouches] anyObject];
-        originalLocation = touchLocation = [touch locationInView:self];
-        touchMoved = false;
-        if ([[event allTouches] count] == 1 && !isDragging) {
-            dragTimer = [NSTimer scheduledTimerWithTimeInterval:0.650
-                                                     target:self
-                                                   selector:@selector(onDragStart:)
-                                                   userInfo:nil
-                                                    repeats:NO];
+        if ([[event allTouches] count] == 3) {
+            if (isInputingText) {
+                Log(LOG_D, @"Closing the keyboard");
+                [keyInputField resignFirstResponder];
+                isInputingText = false;
+            } else {
+                Log(LOG_D, @"Opening the keyboard");
+                // Prepare the textbox used to capture keyboard events.
+                keyInputField.delegate = self;
+                keyInputField.text = @"0";
+                [keyInputField becomeFirstResponder];
+                [keyInputField addTarget:self action:@selector(onKeyboardPressed:) forControlEvents:UIControlEventEditingChanged];
+                
+                // Undo causes issues for our state management, so turn it off
+                [keyInputField.undoManager disableUndoRegistration];
+                
+                isInputingText = true;
+            }
+        }
+        else {
+            [touchHandler touchesBegan:touches withEvent:event];
         }
     }
 }
 
-- (void)onDragStart:(NSTimer*)timer {
-    if (!touchMoved && !isDragging){
-        isDragging = true;
-        LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
+- (BOOL)handleMouseButtonEvent:(int)buttonAction forTouches:(NSSet *)touches withEvent:(UIEvent *)event {
+#if !TARGET_OS_TV
+    if (@available(iOS 13.4, *)) {
+        UITouch* touch = [touches anyObject];
+        if (touch.type == UITouchTypeIndirectPointer) {
+            if (@available(iOS 14.0, *)) {
+                if ([GCMouse current] != nil) {
+                    // We'll handle this with GCMouse. Do nothing here.
+                    return YES;
+                }
+            }
+            
+            UIEventButtonMask normalizedButtonMask;
+            
+            // iOS 14 includes the released button in the buttonMask for the release
+            // event, while iOS 13 does not. Normalize that behavior here.
+            if (@available(iOS 14.0, *)) {
+                if (buttonAction == BUTTON_ACTION_RELEASE) {
+                    normalizedButtonMask = lastMouseButtonMask & ~event.buttonMask;
+                }
+                else {
+                    normalizedButtonMask = event.buttonMask;
+                }
+            }
+            else {
+                normalizedButtonMask = event.buttonMask;
+            }
+            
+            UIEventButtonMask changedButtons = lastMouseButtonMask ^ normalizedButtonMask;
+                        
+            for (int i = BUTTON_LEFT; i <= BUTTON_X2; i++) {
+                UIEventButtonMask buttonFlag;
+                
+                switch (i) {
+                    // Right and Middle are reversed from what iOS uses
+                    case BUTTON_RIGHT:
+                        buttonFlag = UIEventButtonMaskForButtonNumber(2);
+                        break;
+                    case BUTTON_MIDDLE:
+                        buttonFlag = UIEventButtonMaskForButtonNumber(3);
+                        break;
+                        
+                    default:
+                        buttonFlag = UIEventButtonMaskForButtonNumber(i);
+                        break;
+                }
+                
+                if (changedButtons & buttonFlag) {
+                    LiSendMouseButtonEvent(buttonAction, i);
+                }
+            }
+            
+            lastMouseButtonMask = normalizedButtonMask;
+            return YES;
+        }
     }
+#endif
+    
+    return NO;
 }
 
 - (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
-    if (![onScreenControls handleTouchMovedEvent:touches]) {
-        if ([[event allTouches] count] == 1) {
-            UITouch *touch = [[event allTouches] anyObject];
-            CGPoint currentLocation = [touch locationInView:self];
-            
-            if (touchLocation.x != currentLocation.x ||
-                touchLocation.y != currentLocation.y)
-            {
-                int deltaX = currentLocation.x - touchLocation.x;
-                int deltaY = currentLocation.y - touchLocation.y;
-                
-                deltaX *= xDeltaFactor * screenFactor;
-                deltaY *= yDeltaFactor * screenFactor;
-                
-                if (deltaX != 0 || deltaY != 0) {
-                    LiSendMouseMoveEvent(deltaX, deltaY);
-                    touchLocation = currentLocation;
-                    
-                    // If we've moved far enough to confirm this wasn't just human/machine error,
-                    // mark it as such.
-                    if ([self isConfirmedMove:touchLocation from:originalLocation]) {
-                        touchMoved = true;
-                    }
+#if !TARGET_OS_TV
+    if (@available(iOS 13.4, *)) {
+        UITouch *touch = [touches anyObject];
+        if (touch.type == UITouchTypeIndirectPointer) {
+            if (@available(iOS 14.0, *)) {
+                if ([GCMouse current] != nil) {
+                    // We'll handle this with GCMouse. Do nothing here.
+                    return;
                 }
             }
-        } else if ([[event allTouches] count] == 2) {
-            CGPoint firstLocation = [[[[event allTouches] allObjects] objectAtIndex:0] locationInView:self];
-            CGPoint secondLocation = [[[[event allTouches] allObjects] objectAtIndex:1] locationInView:self];
             
-            CGPoint avgLocation = CGPointMake((firstLocation.x + secondLocation.x) / 2, (firstLocation.y + secondLocation.y) / 2);
-            if (touchLocation.y != avgLocation.y) {
-                LiSendScrollEvent(avgLocation.y - touchLocation.y);
-            }
+            // We must handle this event to properly support
+            // drags while the middle, X1, or X2 mouse buttons are
+            // held down. For some reason, left and right buttons
+            // don't require this, but we do it anyway for them too.
+            // Cursor movement without a button held down is handled
+            // in pointerInteraction:regionForRequest:defaultRegion.
+            [self updateCursorLocation:[touch locationInView:self] isMouse:YES];
+            return;
+        }
+    }
+#endif
+    
+    hasUserInteracted = YES;
+    
+    if (![onScreenControls handleTouchMovedEvent:touches]) {
+        [touchHandler touchesMoved:touches withEvent:event];
+    }
+}
 
-            // If we've moved far enough to confirm this wasn't just human/machine error,
-            // mark it as such.
-            if ([self isConfirmedMove:firstLocation from:originalLocation]) {
-                touchMoved = true;
+- (void)pressesBegan:(NSSet<UIPress *> *)presses withEvent:(UIPressesEvent *)event {
+    BOOL handled = NO;
+    
+    if (@available(iOS 13.4, tvOS 13.4, *)) {
+        for (UIPress* press in presses) {
+            // For now, we'll treated it as handled if we handle at least one of the
+            // UIPress events inside the set.
+            if (press.key != nil && [KeyboardSupport sendKeyEvent:press.key down:YES]) {
+                // This will prevent the legacy UITextField from receiving the event
+                handled = YES;
             }
-            
-            touchLocation = avgLocation;
         }
     }
     
+    if (!handled) {
+        [super pressesBegan:presses withEvent:event];
+    }
+}
+
+- (void)pressesEnded:(NSSet<UIPress *> *)presses withEvent:(UIPressesEvent *)event {
+    BOOL handled = NO;
+    
+    if (@available(iOS 13.4, tvOS 13.4, *)) {
+        for (UIPress* press in presses) {
+            // For now, we'll treated it as handled if we handle at least one of the
+            // UIPress events inside the set.
+            if (press.key != nil && [KeyboardSupport sendKeyEvent:press.key down:NO]) {
+                // This will prevent the legacy UITextField from receiving the event
+                handled = YES;
+            }
+        }
+    }
+    
+    if (!handled) {
+        [super pressesEnded:presses withEvent:event];
+    }
 }
 
 - (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
+    if ([self handleMouseButtonEvent:BUTTON_ACTION_RELEASE
+                          forTouches:touches
+                           withEvent:event]) {
+        // If it's a mouse event, we're done
+        return;
+    }
+    
     Log(LOG_D, @"Touch up");
+    
+    hasUserInteracted = YES;
+    
     if (![onScreenControls handleTouchUpEvent:touches]) {
-        [dragTimer invalidate];
-        dragTimer = nil;
-        if (isDragging) {
-            isDragging = false;
-            LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
-        } else if (!touchMoved) {
-            if ([[event allTouches] count] == 3) {
-                if (isInputingText) {
-                    Log(LOG_D, @"Closing the keyboard");
-                    [_keyInputField resignFirstResponder];
-                    isInputingText = false;
-                } else {
-                    Log(LOG_D, @"Opening the keyboard");
-                    // Prepare the textbox used to capture keyboard events.
-                    _keyInputField.delegate = self;
-                    _keyInputField.text = @"0";
-                    [_keyInputField becomeFirstResponder];
-                    [_keyInputField addTarget:self action:@selector(onKeyboardPressed:) forControlEvents:UIControlEventEditingChanged];
-                    
-                    // Undo causes issues for our state management, so turn it off
-                    [_keyInputField.undoManager disableUndoRegistration];
-                    
-                    isInputingText = true;
-                }
-            } else if ([[event allTouches] count]  == 2) {
-                dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
-                    Log(LOG_D, @"Sending right mouse button press");
-                    
-                    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_RIGHT);
-                    
-                    // Wait 100 ms to simulate a real button press
-                    usleep(100 * 1000);
-                    
-                    LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_RIGHT);
-                });
-            } else if ([[event allTouches] count]  == 1) {
-                dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
-                    if (!self->isDragging){
-                        Log(LOG_D, @"Sending left mouse button press");
-                        
-                        LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, BUTTON_LEFT);
-                        
-                        // Wait 100 ms to simulate a real button press
-                        usleep(100 * 1000);
-                    }
-                    self->isDragging = false;
-                    LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, BUTTON_LEFT);
-                });
-            }
-        }
-        
-        // We we're moving from 2+ touches to 1. Synchronize the current position
-        // of the active finger so we don't jump unexpectedly on the next touchesMoved
-        // callback when finger 1 switches on us.
-        if ([[event allTouches] count] - [touches count] == 1) {
-            NSMutableSet *activeSet = [[NSMutableSet alloc] initWithCapacity:[[event allTouches] count]];
-            [activeSet unionSet:[event allTouches]];
-            [activeSet minusSet:touches];
-            touchLocation = [[activeSet anyObject] locationInView:self];
-            
-            // Mark this touch as moved so we don't send a left mouse click if the user
-            // right clicks without moving their other finger.
-            touchMoved = true;
-        }
+        [touchHandler touchesEnded:touches withEvent:event];
     }
 }
 
 - (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
+    [touchHandler touchesCancelled:touches withEvent:event];
+    [self handleMouseButtonEvent:BUTTON_ACTION_RELEASE
+                      forTouches:touches
+                       withEvent:event];
+}
+
+#if !TARGET_OS_TV
+- (void) updateCursorLocation:(CGPoint)location isMouse:(BOOL)isMouse {
+    // These are now relative to the StreamView, however we need to scale them
+    // further to make them relative to the actual video portion.
+    float x = location.x - self.bounds.origin.x;
+    float y = location.y - self.bounds.origin.y;
+    
+    // For some reason, we don't seem to always get to the bounds of the window
+    // so we'll subtract 1 pixel if we're to the left/below of the origin and
+    // and add 1 pixel if we're to the right/above. It should be imperceptible
+    // to the user but it will allow activation of gestures that require contact
+    // with the edge of the screen (like Aero Snap).
+    if (x < self.bounds.size.width / 2) {
+        x--;
+    }
+    else {
+        x++;
+    }
+    if (y < self.bounds.size.height / 2) {
+        y--;
+    }
+    else {
+        y++;
+    }
+    
+    // This logic mimics what iOS does with AVLayerVideoGravityResizeAspect
+    CGSize videoSize;
+    CGPoint videoOrigin;
+    if (self.bounds.size.width > self.bounds.size.height * streamAspectRatio) {
+        videoSize = CGSizeMake(self.bounds.size.height * streamAspectRatio, self.bounds.size.height);
+    } else {
+        videoSize = CGSizeMake(self.bounds.size.width, self.bounds.size.width / streamAspectRatio);
+    }
+    videoOrigin = CGPointMake(self.bounds.size.width / 2 - videoSize.width / 2,
+                              self.bounds.size.height / 2 - videoSize.height / 2);
+    
+    // Confine the cursor to the video region. We don't just discard events outside
+    // the region because we won't always get one exactly when the mouse leaves the region.
+    x = MIN(MAX(x, videoOrigin.x), videoOrigin.x + videoSize.width);
+    y = MIN(MAX(y, videoOrigin.y), videoOrigin.y + videoSize.height);
+    
+    // Send the mouse position relative to the video region if it has changed
+    // if we're receiving coordinates from a real mouse.
+    //
+    // NB: It is important for functionality (not just optimization) to only
+    // send it if the value has changed. We will receive one of these events
+    // any time the user presses a modifier key, which can result in errant
+    // mouse motion when using a Citrix X1 mouse.
+    if (x != lastMouseX || y != lastMouseY || !isMouse) {
+        if (lastMouseX != 0 || lastMouseY != 0 || !isMouse) {
+            LiSendMousePositionEvent(x - videoOrigin.x, y - videoOrigin.y,
+                                     videoSize.width, videoSize.height);
+        }
+        
+        if (isMouse) {
+            lastMouseX = x;
+            lastMouseY = y;
+        }
+    }
+}
+
+- (UIPointerRegion *)pointerInteraction:(UIPointerInteraction *)interaction
+                       regionForRequest:(UIPointerRegionRequest *)request
+                          defaultRegion:(UIPointerRegion *)defaultRegion API_AVAILABLE(ios(13.4)) {
+    if (@available(iOS 14.0, *)) {
+        if ([GCMouse current] != nil) {
+            // We'll handle this with GCMouse. Do nothing here.
+            return nil;
+        }
+    }
+    
+    // This logic mimics what iOS does with AVLayerVideoGravityResizeAspect
+    CGSize videoSize;
+    CGPoint videoOrigin;
+    if (self.bounds.size.width > self.bounds.size.height * streamAspectRatio) {
+        videoSize = CGSizeMake(self.bounds.size.height * streamAspectRatio, self.bounds.size.height);
+    } else {
+        videoSize = CGSizeMake(self.bounds.size.width, self.bounds.size.width / streamAspectRatio);
+    }
+    videoOrigin = CGPointMake(self.bounds.size.width / 2 - videoSize.width / 2,
+                              self.bounds.size.height / 2 - videoSize.height / 2);
+    
+    // Move the cursor on the host if no buttons are pressed.
+    // Motion with buttons pressed in handled in touchesMoved:
+    if (lastMouseButtonMask == 0) {
+        [self updateCursorLocation:request.location isMouse:YES];
+    }
+    
+    // The pointer interaction should cover the video region only
+    return [UIPointerRegion regionWithRect:CGRectMake(videoOrigin.x, videoOrigin.y, videoSize.width, videoSize.height) identifier:nil];
+}
+
+- (UIPointerStyle *)pointerInteraction:(UIPointerInteraction *)interaction styleForRegion:(UIPointerRegion *)region  API_AVAILABLE(ios(13.4)) {
+    // Always hide the mouse cursor over our stream view
+    return [UIPointerStyle hiddenPointerStyle];
+}
+
+- (void)mouseWheelMovedContinuous:(UIPanGestureRecognizer *)gesture {
+    switch (gesture.state) {
+        case UIGestureRecognizerStateBegan:
+        case UIGestureRecognizerStateChanged:
+            break;
+        
+        case UIGestureRecognizerStateEnded:
+        default:
+            // Ignore recognition failure and other states
+            lastScrollTranslation = CGPointMake(0, 0);
+            return;
+    }
+    
+    CGPoint currentScrollTranslation = [gesture translationInView:self];
+    short translationDeltaY = ((currentScrollTranslation.y - lastScrollTranslation.y) / self.bounds.size.height) * 120; // WHEEL_DELTA
+    if (translationDeltaY != 0) {
+        LiSendHighResScrollEvent(translationDeltaY * 20);
+        lastScrollTranslation = currentScrollTranslation;
+    }
+}
+
+- (void)mouseWheelMovedDiscrete:(UIPanGestureRecognizer *)gesture {
+    switch (gesture.state) {
+        case UIGestureRecognizerStateBegan:
+        case UIGestureRecognizerStateChanged:
+            break;
+        
+        case UIGestureRecognizerStateEnded:
+        default:
+            // Ignore recognition failure and other states
+            lastScrollTranslation = CGPointMake(0, 0);
+            return;
+    }
+    
+    // Using velocityInView is 0 for discrete scroll events
+    // when scrolling very slowly, but translationInView does work.
+    CGPoint currentScrollTranslation = [gesture translationInView:self];
+    short translationDeltaY = currentScrollTranslation.y - lastScrollTranslation.y;
+    if (translationDeltaY != 0) {
+        LiSendScrollEvent(translationDeltaY > 0 ? 1 : -1);
+    }
+    
+    lastScrollTranslation = currentScrollTranslation;
+}
+
+#endif
+
+- (BOOL)gestureRecognizerShouldBegin:(UIGestureRecognizer *)gestureRecognizer {
+    if (@available(iOS 13.0, *)) {
+        // Disable the 3 finger tap gestures that trigger the copy/paste/undo toolbar on iOS 13+
+        return gestureRecognizer.name == nil || ![gestureRecognizer.name hasPrefix:@"kbProductivity."];
+    }
+    else {
+        return YES;
+    }
 }
 
 - (BOOL)textFieldShouldReturn:(UITextField *)textField {
@@ -217,13 +530,24 @@
             LiSendKeyboardEvent(0x08, KEY_ACTION_UP, 0);
         } else {
             // Character 0 will be our known sentinel value
+            
+            // Check if any characters exist which can't be represented in a basic key event
             for (int i = 1; i < [inputText length]; i++) {
                 struct KeyEvent event = [KeyboardSupport translateKeyEvent:[inputText characterAtIndex:i] withModifierFlags:0];
                 if (event.keycode == 0) {
-                    // If we don't know the code, don't send anything.
-                    Log(LOG_W, @"Unknown key code: [%c]", [inputText characterAtIndex:i]);
-                    continue;
+                    // We found an unknown key, so send the entire string as UTF-8
+                    const char* utf8String = [inputText UTF8String];
+                    
+                    // Skip the first character which is our sentinel
+                    LiSendUtf8TextEvent(utf8String + 1, (int)strlen(utf8String) - 1);
+                    return;
                 }
+            }
+            
+            // We didn't find any unknown characters, so send them all as basic key events
+            for (int i = 1; i < [inputText length]; i++) {
+                struct KeyEvent event = [KeyboardSupport translateKeyEvent:[inputText characterAtIndex:i] withModifierFlags:0];
+                assert(event.keycode != 0);
                 [self sendLowLevelEvent:event];
             }
         }
@@ -311,4 +635,57 @@
     return commands;
 }
 
+- (void)connectedStateDidChangeWithIdentifier:(NSUUID * _Nonnull)identifier isConnected:(BOOL)isConnected {
+    NSLog(@"Citrix X1 mouse state change: %@ -> %s",
+          identifier, isConnected ? "connected" : "disconnected");
+}
+
+- (void)mouseDidMoveWithIdentifier:(NSUUID * _Nonnull)identifier deltaX:(int16_t)deltaX deltaY:(int16_t)deltaY {
+    accumulatedMouseDeltaX += deltaX / X1_MOUSE_SPEED_DIVISOR;
+    accumulatedMouseDeltaY += deltaY / X1_MOUSE_SPEED_DIVISOR;
+    
+    short shortX = (short)accumulatedMouseDeltaX;
+    short shortY = (short)accumulatedMouseDeltaY;
+    
+    if (shortX == 0 && shortY == 0) {
+        return;
+    }
+    
+    LiSendMouseMoveEvent(shortX, shortY);
+    
+    accumulatedMouseDeltaX -= shortX;
+    accumulatedMouseDeltaY -= shortY;
+}
+
+- (int) buttonFromX1ButtonCode:(enum X1MouseButton)button {
+    switch (button) {
+        case X1MouseButtonLeft:
+            return BUTTON_LEFT;
+        case X1MouseButtonRight:
+            return BUTTON_RIGHT;
+        case X1MouseButtonMiddle:
+            return BUTTON_MIDDLE;
+        default:
+            return -1;
+    }
+}
+
+- (void)mouseDownWithIdentifier:(NSUUID * _Nonnull)identifier button:(enum X1MouseButton)button {
+    LiSendMouseButtonEvent(BUTTON_ACTION_PRESS, [self buttonFromX1ButtonCode:button]);
+}
+
+- (void)mouseUpWithIdentifier:(NSUUID * _Nonnull)identifier button:(enum X1MouseButton)button {
+    LiSendMouseButtonEvent(BUTTON_ACTION_RELEASE, [self buttonFromX1ButtonCode:button]);
+}
+
+- (void)wheelDidScrollWithIdentifier:(NSUUID * _Nonnull)identifier deltaZ:(int8_t)deltaZ {
+    LiSendScrollEvent(deltaZ);
+}
+
+#if !TARGET_OS_TV
+- (BOOL)isMultipleTouchEnabled {
+    return YES;
+}
+#endif
+
 @end

Limelight/Limelight-Info.plist

@@ -4,6 +4,8 @@
 <dict>
 	<key>CADisableMinimumFrameDuration</key>
 	<true/>
+	<key>CADisableMinimumFrameDurationOnPhone</key>
+	<true/>
 	<key>CFBundleDevelopmentRegion</key>
 	<string>en</string>
 	<key>CFBundleDisplayName</key>
@@ -19,11 +21,11 @@
 	<key>CFBundlePackageType</key>
 	<string>APPL</string>
 	<key>CFBundleShortVersionString</key>
-	<string>2.6.0</string>
+	<string>$(MARKETING_VERSION)</string>
 	<key>CFBundleSignature</key>
 	<string>????</string>
 	<key>CFBundleVersion</key>
-	<string>1</string>
+	<string>$(CURRENT_PROJECT_VERSION)</string>
 	<key>GCSupportedGameControllers</key>
 	<array>
 		<dict>
@@ -37,6 +39,8 @@
 	</array>
 	<key>GCSupportsControllerUserInteraction</key>
 	<true/>
+	<key>ITSAppUsesNonExemptEncryption</key>
+	<false/>
 	<key>LSApplicationCategoryType</key>
 	<string>public.app-category.games</string>
 	<key>LSRequiresIPhoneOS</key>
@@ -46,6 +50,18 @@
 		<key>NSAllowsArbitraryLoads</key>
 		<true/>
 	</dict>
+	<key>NSBluetoothAlwaysUsageDescription</key>
+	<string>Bluetooth access allows Moonlight to connect to Citrix X1 mice.</string>
+	<key>NSBluetoothPeripheralUsageDescription</key>
+	<string>Bluetooth access allows Moonlight to connect to Citrix X1 mice.</string>
+	<key>NSBonjourServices</key>
+	<array>
+		<string>_nvstream._tcp</string>
+	</array>
+	<key>NSLocalNetworkUsageDescription</key>
+	<string>Moonlight uses the local network to connect to your gaming PC for streaming.</string>
+	<key>UIApplicationSupportsIndirectInputEvents</key>
+	<true/>
 	<key>UILaunchStoryboardName</key>
 	<string>Launch Screen</string>
 	<key>UIMainStoryboardFile</key>

Limelight/Limelight-Prefix.pch

@@ -21,5 +21,4 @@
 #import <Foundation/Foundation.h>
 #import <CoreData/CoreData.h>
 #import "Logger.h"
-#include "OSPortabilityDefs.h"
 #endif

Limelight/Limelight.xcdatamodeld/.xccurrentversion

@@ -3,6 +3,6 @@
 <plist version="1.0">
 <dict>
 	<key>_XCCurrentVersionName</key>
-	<string>Moonlight v1.3.xcdatamodel</string>
+	<string>Moonlight v1.6.xcdatamodel</string>
 </dict>
 </plist>

Limelight/Limelight.xcdatamodeld/Moonlight v1.4.xcdatamodel/contents

@@ -0,0 +1,41 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<model type="com.apple.IDECoreDataModeler.DataModel" documentVersion="1.0" lastSavedToolsVersion="14490.99" systemVersion="18F132" minimumToolsVersion="Xcode 7.3" sourceLanguage="Objective-C" userDefinedModelVersionIdentifier="">
+    <entity name="App" representedClassName="App" syncable="YES" codeGenerationType="class">
+        <attribute name="hdrSupported" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="id" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <relationship name="host" maxCount="1" deletionRule="Nullify" destinationEntity="Host" inverseName="appList" inverseEntity="Host" syncable="YES"/>
+    </entity>
+    <entity name="Host" representedClassName="Host" syncable="YES" codeGenerationType="class">
+        <attribute name="address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="externalAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="ipv6Address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="localAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="mac" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <attribute name="pairState" optional="YES" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="serverCert" optional="YES" attributeType="Binary" syncable="YES"/>
+        <attribute name="serverCodecModeSupport" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uuid" optional="YES" attributeType="String" syncable="YES"/>
+        <relationship name="appList" optional="YES" toMany="YES" deletionRule="Cascade" destinationEntity="App" inverseName="host" inverseEntity="App" syncable="YES"/>
+    </entity>
+    <entity name="Settings" representedClassName="Settings" syncable="YES" codeGenerationType="class">
+        <attribute name="bitrate" attributeType="Integer 32" defaultValueString="10000" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="enableHdr" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="framerate" attributeType="Integer 32" defaultValueString="60" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="height" attributeType="Integer 32" defaultValueString="720" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="multiController" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="onscreenControls" attributeType="Integer 32" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="optimizeGames" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="playAudioOnPC" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="streamingRemotely" attributeType="Boolean" defaultValueString="0" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uniqueId" attributeType="String" syncable="YES"/>
+        <attribute name="useHevc" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="width" attributeType="Integer 32" defaultValueString="1280" usesScalarValueType="NO" syncable="YES"/>
+    </entity>
+    <elements>
+        <element name="App" positionX="0" positionY="54" width="128" height="105"/>
+        <element name="Host" positionX="0" positionY="0" width="128" height="210"/>
+        <element name="Settings" positionX="0" positionY="0" width="128" height="225"/>
+    </elements>
+</model>

Limelight/Limelight.xcdatamodeld/Moonlight v1.5.xcdatamodel/contents

@@ -0,0 +1,42 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<model type="com.apple.IDECoreDataModeler.DataModel" documentVersion="1.0" lastSavedToolsVersion="16119" systemVersion="19E266" minimumToolsVersion="Xcode 7.3" sourceLanguage="Objective-C" userDefinedModelVersionIdentifier="">
+    <entity name="App" representedClassName="App" syncable="YES" codeGenerationType="class">
+        <attribute name="hdrSupported" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="id" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <relationship name="host" maxCount="1" deletionRule="Nullify" destinationEntity="Host" inverseName="appList" inverseEntity="Host" syncable="YES"/>
+    </entity>
+    <entity name="Host" representedClassName="Host" syncable="YES" codeGenerationType="class">
+        <attribute name="address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="externalAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="ipv6Address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="localAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="mac" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <attribute name="pairState" optional="YES" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="serverCert" optional="YES" attributeType="Binary" syncable="YES"/>
+        <attribute name="serverCodecModeSupport" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uuid" optional="YES" attributeType="String" syncable="YES"/>
+        <relationship name="appList" optional="YES" toMany="YES" deletionRule="Cascade" destinationEntity="App" inverseName="host" inverseEntity="App" syncable="YES"/>
+    </entity>
+    <entity name="Settings" representedClassName="Settings" syncable="YES" codeGenerationType="class">
+        <attribute name="bitrate" attributeType="Integer 32" defaultValueString="10000" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="btMouseSupport" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="enableHdr" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="framerate" attributeType="Integer 32" defaultValueString="60" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="height" attributeType="Integer 32" defaultValueString="720" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="multiController" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="onscreenControls" attributeType="Integer 32" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="optimizeGames" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="playAudioOnPC" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="streamingRemotely" attributeType="Boolean" defaultValueString="0" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uniqueId" attributeType="String" syncable="YES"/>
+        <attribute name="useHevc" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="width" attributeType="Integer 32" defaultValueString="1280" usesScalarValueType="NO" syncable="YES"/>
+    </entity>
+    <elements>
+        <element name="App" positionX="0" positionY="54" width="128" height="105"/>
+        <element name="Host" positionX="0" positionY="0" width="128" height="210"/>
+        <element name="Settings" positionX="0" positionY="0" width="128" height="238"/>
+    </elements>
+</model>

Limelight/Limelight.xcdatamodeld/Moonlight v1.6.xcdatamodel/contents

@@ -0,0 +1,44 @@
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<model type="com.apple.IDECoreDataModeler.DataModel" documentVersion="1.0" lastSavedToolsVersion="17192" systemVersion="19H2" minimumToolsVersion="Xcode 7.3" sourceLanguage="Objective-C" userDefinedModelVersionIdentifier="">
+    <entity name="App" representedClassName="App" syncable="YES" codeGenerationType="class">
+        <attribute name="hdrSupported" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="hidden" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="id" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <relationship name="host" maxCount="1" deletionRule="Nullify" destinationEntity="Host" inverseName="appList" inverseEntity="Host" syncable="YES"/>
+    </entity>
+    <entity name="Host" representedClassName="Host" syncable="YES" codeGenerationType="class">
+        <attribute name="address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="externalAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="ipv6Address" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="localAddress" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="mac" optional="YES" attributeType="String" syncable="YES"/>
+        <attribute name="name" attributeType="String" syncable="YES"/>
+        <attribute name="pairState" optional="YES" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="serverCert" optional="YES" attributeType="Binary" syncable="YES"/>
+        <attribute name="serverCodecModeSupport" attributeType="Integer 32" defaultValueString="0" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uuid" optional="YES" attributeType="String" syncable="YES"/>
+        <relationship name="appList" optional="YES" toMany="YES" deletionRule="Cascade" destinationEntity="App" inverseName="host" inverseEntity="App" syncable="YES"/>
+    </entity>
+    <entity name="Settings" representedClassName="Settings" syncable="YES" codeGenerationType="class">
+        <attribute name="absoluteTouchMode" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="bitrate" attributeType="Integer 32" defaultValueString="10000" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="btMouseSupport" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="enableHdr" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="framerate" attributeType="Integer 32" defaultValueString="60" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="height" attributeType="Integer 32" defaultValueString="720" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="multiController" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="onscreenControls" attributeType="Integer 32" defaultValueString="1" usesScalarValueType="NO" syncable="YES"/>
+        <attribute name="optimizeGames" attributeType="Boolean" defaultValueString="YES" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="playAudioOnPC" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="statsOverlay" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="uniqueId" attributeType="String" syncable="YES"/>
+        <attribute name="useHevc" attributeType="Boolean" defaultValueString="NO" usesScalarValueType="YES" syncable="YES"/>
+        <attribute name="width" attributeType="Integer 32" defaultValueString="1280" usesScalarValueType="NO" syncable="YES"/>
+    </entity>
+    <elements>
+        <element name="App" positionX="0" positionY="54" width="128" height="118"/>
+        <element name="Host" positionX="0" positionY="0" width="128" height="210"/>
+        <element name="Settings" positionX="0" positionY="0" width="128" height="268"/>
+    </elements>
+</model>

Limelight/Network/AppAssetResponse.m

@@ -16,13 +16,8 @@
     self.statusMessage = @"App asset has no status message";
     self.statusCode = -1;
 }
-- (OSImage*) getImage {
-#if TARGET_OS_IPHONE
-    OSImage* appImage = [[OSImage alloc] initWithData:self.data];
-#else
-    OSImage* appImage = nil;
-#endif
-    return appImage;
+- (UIImage*) getImage {
+    return [[UIImage alloc] initWithData:self.data];
 }
 
 @end

Limelight/Network/AppAssetRetriever.m

@@ -24,16 +24,12 @@ static const int MAX_ATTEMPTS = 5;
         AppAssetResponse* appAssetResp = [[AppAssetResponse alloc] init];
         [hMan executeRequestSynchronously:[HttpRequest requestForResponse:appAssetResp withUrlRequest:[hMan newAppAssetRequestWithAppId:self.app.id]]];
 
-#if TARGET_OS_IPHONE
         if (appAssetResp.data != nil) {
             NSString* boxArtPath = [AppAssetManager boxArtPathForApp:self.app];
             [[NSFileManager defaultManager] createDirectoryAtPath:[boxArtPath stringByDeletingLastPathComponent] withIntermediateDirectories:YES attributes:nil error:nil];
             [appAssetResp.data writeToFile:boxArtPath atomically:NO];
             break;
         }
-#else
-        
-#endif
         
         if (![self isCancelled]) {
             [NSThread sleepForTimeInterval:RETRY_DELAY];

Limelight/Network/AppListResponse.m

@@ -20,6 +20,7 @@ static const char* TAG_APP = "App";
 static const char* TAG_APP_TITLE = "AppTitle";
 static const char* TAG_APP_ID = "ID";
 static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
+static const char* TAG_APP_INSTALL_PATH = "AppInstallPath";
 
 - (void)populateWithData:(NSData *)xml {
     self.data = xml;
@@ -43,7 +44,7 @@ static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
     
     xmlChar* statusStr = xmlGetProp(node, (const xmlChar*)[TAG_STATUS_CODE UTF8String]);
     if (statusStr != NULL) {
-        int status = [[NSString stringWithUTF8String:(const char*)statusStr] intValue];
+        int status = (int)[[NSString stringWithUTF8String:(const char*)statusStr] longLongValue];
         xmlFree(statusStr);
         self.statusCode = status;
     }
@@ -68,6 +69,7 @@ static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
             NSString* appName = @"";
             NSString* appId = nil;
             NSString* hdrSupported = @"0";
+            NSString* appInstallPath = nil;
             while (appInfoNode != NULL) {
                 if (!xmlStrcmp(appInfoNode->name, (xmlChar*)TAG_APP_TITLE)) {
                     xmlChar* nodeVal = xmlNodeListGetString(docPtr, appInfoNode->xmlChildrenNode, 1);
@@ -87,7 +89,14 @@ static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
                         hdrSupported = [[NSString alloc] initWithCString:(const char*)nodeVal encoding:NSUTF8StringEncoding];
                         xmlFree(nodeVal);
                     }
+                } else if (!xmlStrcmp(appInfoNode->name, (xmlChar*)TAG_APP_INSTALL_PATH)) {
+                    xmlChar* nodeVal = xmlNodeListGetString(docPtr, appInfoNode->xmlChildrenNode, 1);
+                    if (nodeVal != NULL) {
+                        appInstallPath = [[NSString alloc] initWithCString:(const char*)nodeVal encoding:NSUTF8StringEncoding];
+                        xmlFree(nodeVal);
+                    }
                 }
+
                 appInfoNode = appInfoNode->next;
             }
             if (appId != nil) {
@@ -95,6 +104,7 @@ static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
                 app.name = appName;
                 app.id = appId;
                 app.hdrSupported = [hdrSupported intValue] != 0;
+                app.installPath = appInstallPath;
                 [_appList addObject:app];
             }
         }
@@ -102,6 +112,42 @@ static const char* TAG_HDR_SUPPORTED = "IsHdrSupported";
     }
     
     xmlFreeDoc(docPtr);
+    
+#ifdef ENABLE_APP_STORE_RESTRICTIONS
+    // APP STORE REVIEW COMPLIANCE
+    //
+    // Remove default Steam entry from the app list to comply with Apple App Store Guideline 4.2.7d:
+    //
+    // The UI appearing on the client does not resemble an iOS or App Store view, does not provide a store-like interface,
+    // or include the ability to browse, select, or purchase software not already owned or licensed by the user.
+    //
+    // However, if the user manually adds Steam themselves, then we will display it.
+    TemporaryApp* officialSteamApp = nil;
+    TemporaryApp* manuallyAddedSteamApp = nil;
+    for (TemporaryApp* app in _appList) {
+        if (app.installPath != nil && [[app.installPath lowercaseString] hasSuffix:@"\\steam\\"]) {
+            // The official Steam app is marked as HDR supported, while manually added ones are not.
+            if ([app.name isEqualToString:@"Steam"] && app.hdrSupported) {
+                officialSteamApp = app;
+            }
+            else {
+                manuallyAddedSteamApp = app;
+            }
+        }
+    }
+    
+    // To be safe, don't do anything if we didn't find an HDR-enabled Steam app.
+    if (officialSteamApp != nil) {
+        // If we didn't find a manually added Steam app, remove the official one to
+        // comply with the App Store guidelines.
+        if (manuallyAddedSteamApp == nil) {
+            [_appList removeObject:officialSteamApp];
+        }
+        else {
+            [_appList removeObject:manuallyAddedSteamApp];
+        }
+    }
+#endif
 }
 
 - (NSSet*) getAppList {

Limelight/Network/ConnectionHelper.m

@@ -18,11 +18,11 @@
     HttpManager* hMan = [[HttpManager alloc] initWithHost:hostIP uniqueId:uniqueId serverCert:cert];
     
     // Try up to 5 times to get the app list
-    AppListResponse* appListResp;
+    AppListResponse* appListResp = nil;
     for (int i = 0; i < 5; i++) {
         appListResp = [[AppListResponse alloc] init];
         [hMan executeRequestSynchronously:[HttpRequest requestForResponse:appListResp withUrlRequest:[hMan newAppListRequest]]];
-        if (appListResp == nil || ![appListResp isStatusOk] || [appListResp getAppList] == nil) {
+        if (![appListResp isStatusOk] || [appListResp getAppList] == nil) {
             Log(LOG_W, @"Failed to get applist on try %d: %@", i, appListResp.statusMessage);
             
             // Wait for one second then retry
@@ -30,10 +30,11 @@
         }
         else {
             Log(LOG_I, @"App list successfully retreived - took %d tries", i);
-            return appListResp;
+            break;
         }
     }
-    return nil;
+    
+    return appListResp;
 }
 
 @end

Limelight/Network/DiscoveryManager.h

@@ -21,8 +21,11 @@
 - (void) startDiscovery;
 - (void) stopDiscovery;
 - (void) stopDiscoveryBlocking;
+- (void) resetDiscoveryState;
 - (BOOL) addHostToDiscovery:(TemporaryHost*)host;
 - (void) removeHostFromDiscovery:(TemporaryHost*)host;
+- (void) pauseDiscoveryForHost:(TemporaryHost *)host;
+- (void) resumeDiscoveryForHost:(TemporaryHost *)host;
 - (void) discoverHost:(NSString*)hostAddress withCallback:(void (^)(TemporaryHost*, NSString*))callback;
 
 @end

Limelight/Network/DiscoveryManager.m

@@ -15,8 +15,14 @@
 #import "ServerInfoResponse.h"
 #import "IdManager.h"
 
+#include <Limelight.h>
+#include <arpa/inet.h>
+#include <sys/socket.h>
+#include <netdb.h>
+
 @implementation DiscoveryManager {
     NSMutableArray* _hostQueue;
+    NSMutableSet* _pausedHosts;
     id<DiscoveryCallback> _callback;
     MDNSManager* _mdnsMan;
     NSOperationQueue* _opQueue;
@@ -33,6 +39,7 @@
     _callback = callback;
     shouldDiscover = NO;
     _hostQueue = [NSMutableArray array];
+    _pausedHosts = [NSMutableSet set];
     for (TemporaryHost* host in hosts)
     {
         [self addHostToDiscovery:host];
@@ -47,26 +54,163 @@
     return self;
 }
 
-- (void) discoverHost:(NSString *)hostAddress withCallback:(void (^)(TemporaryHost *, NSString*))callback {
-    HttpManager* hMan = [[HttpManager alloc] initWithHost:hostAddress uniqueId:_uniqueId serverCert:nil];
++ (BOOL) isAddressLAN:(in_addr_t)addr {
+    addr = htonl(addr);
+    
+    // 10.0.0.0/8
+    if ((addr & 0xFF000000) == 0x0A000000) {
+        return YES;
+    }
+    // 172.16.0.0/12
+    else if ((addr & 0xFFF00000) == 0xAC100000) {
+        return YES;
+    }
+    // 192.168.0.0/16
+    else if ((addr & 0xFFFF0000) == 0xC0A80000) {
+        return YES;
+    }
+    // 169.254.0.0/16
+    else if ((addr & 0xFFFF0000) == 0xA9FE0000) {
+        return YES;
+    }
+    // 100.64.0.0/10 - RFC6598 official CGN address (shouldn't see this in a LAN)
+    else if ((addr & 0xFFC00000) == 0x64400000) {
+        return YES;
+    }
+    
+    return NO;
+}
+
+// This ensures that only RFC 1918 IPv4 addresses can be passed to
+// the Add PC dialog. This is required to comply with Apple App Store
+// Guideline 4.2.7a.
++ (BOOL) isProhibitedAddress:(NSString*)address {
+#ifdef ENABLE_APP_STORE_RESTRICTIONS
+    struct addrinfo hints;
+    struct addrinfo* result;
+    int err;
+    
+    // We're explicitly using AF_INET here because we don't want to
+    // ever receive a synthesized IPv6 address here, even on NAT64.
+    // IPv6 addresses are not restricted here because we cannot easily
+    // tell whether they are local or not.
+    memset(&hints, 0, sizeof(hints));
+    hints.ai_family = AF_INET;
+    err = getaddrinfo([address UTF8String], NULL, &hints, &result);
+    if (err != 0 || result == NULL) {
+        Log(LOG_W, @"getaddrinfo(%@) failed: %d", address, err);
+        return NO;
+    }
+    
+    if (result->ai_family != AF_INET) {
+        // This should never happen due to our hints
+        assert(result->ai_family == AF_INET);
+        Log(LOG_W, @"Unexpected address family: %d", result->ai_family);
+        freeaddrinfo(result);
+        return NO;
+    }
+    
+    BOOL ret = ![DiscoveryManager isAddressLAN:((struct sockaddr_in*)result->ai_addr)->sin_addr.s_addr];
+    freeaddrinfo(result);
+
+    return ret;
+#else
+    return NO;
+#endif
+}
+
+- (ServerInfoResponse*) getServerInfoResponseForAddress:(NSString*)address {
+    HttpManager* hMan = [[HttpManager alloc] initWithHost:address uniqueId:_uniqueId serverCert:nil];
     ServerInfoResponse* serverInfoResponse = [[ServerInfoResponse alloc] init];
     [hMan executeRequestSynchronously:[HttpRequest requestForResponse:serverInfoResponse withUrlRequest:[hMan newServerInfoRequest:false] fallbackError:401 fallbackRequest:[hMan newHttpServerInfoRequest]]];
+    return serverInfoResponse;
+}
+
+- (void) discoverHost:(NSString *)hostAddress withCallback:(void (^)(TemporaryHost *, NSString*))callback {
+    BOOL prohibitedAddress = [DiscoveryManager isProhibitedAddress:hostAddress];
+    NSString* prohibitedAddressMessage = [NSString stringWithFormat: @"Moonlight only supports adding PCs on your local network on %s.",
+    #if TARGET_OS_TV
+                                   "tvOS"
+    #else
+                                   "iOS"
+    #endif
+                             ];
+    ServerInfoResponse* serverInfoResponse = [self getServerInfoResponseForAddress:hostAddress];
     
     TemporaryHost* host = nil;
     if ([serverInfoResponse isStatusOk]) {
         host = [[TemporaryHost alloc] init];
         host.activeAddress = host.address = hostAddress;
-        host.online = YES;
+        host.state = StateOnline;
         [serverInfoResponse populateHost:host];
+        
+        // Check if this is a new PC
+        if (![self getHostInDiscovery:host.uuid]) {
+            // Enforce LAN restriction for App Store Guideline 4.2.7a
+            if ([DiscoveryManager isProhibitedAddress:hostAddress]) {
+                // We have a prohibited address. This might be because the user specified their WAN address
+                // instead of their LAN address. If that's the case, we'll try their LAN address and if we
+                // can reach it through that address, we'll allow it.
+                ServerInfoResponse* lanInfo = [self getServerInfoResponseForAddress:host.localAddress];
+                if ([lanInfo isStatusOk]) {
+                    TemporaryHost* lanHost = [[TemporaryHost alloc] init];
+                    [lanInfo populateHost:lanHost];
+                    
+                    if (![lanHost.uuid isEqualToString:host.uuid]) {
+                        // This is a different host, so it's prohibited
+                        prohibitedAddress = YES;
+                    }
+                    else {
+                        // This is the same host that is reachable on the LAN
+                        prohibitedAddress = NO;
+                    }
+                }
+                else {
+                    // LAN request failed, so it's a prohibited address
+                    prohibitedAddress = YES;
+                }
+            }
+            else {
+                // It's an RFC 1918 IPv4 address or IPv6 address which counts as LAN
+                prohibitedAddress = NO;
+            }
+            
+            if (prohibitedAddress) {
+                callback(nil, prohibitedAddressMessage);
+                return;
+            }
+            else if ([DiscoveryManager isAddressLAN:inet_addr([hostAddress UTF8String])]) {
+                // Don't send a STUN request if we're connected to a VPN. We'll likely get the VPN
+                // gateway's external address rather than the external address of the LAN.
+                if (![Utils isActiveNetworkVPN]) {
+                    // This host was discovered over a permissible LAN address, so we can update our
+                    // external address for this host.
+                    struct in_addr wanAddr;
+                    int err = LiFindExternalAddressIP4("stun.moonlight-stream.org", 3478, &wanAddr.s_addr);
+                    if (err == 0) {
+                        char addrStr[INET_ADDRSTRLEN];
+                        inet_ntop(AF_INET, &wanAddr, addrStr, sizeof(addrStr));
+                        host.externalAddress = [NSString stringWithFormat: @"%s", addrStr];
+                    }
+                }
+            }
+        }
+        
         if (![self addHostToDiscovery:host]) {
             callback(nil, @"Host information updated");
         } else {
             callback(host, nil);
         }
-    } else {
+    } else if (!prohibitedAddress) {
         callback(nil, @"Could not connect to host. Ensure GameStream is enabled in GeForce Experience on your PC.");
+    } else {
+        callback(nil, prohibitedAddressMessage);
     }
-    
+}
+
+- (void) resetDiscoveryState {
+    // Allow us to rediscover hosts that were already found before
+    [_mdnsMan forgetHosts];
 }
 
 - (void) startDiscovery {
@@ -77,8 +221,13 @@
     Log(LOG_I, @"Starting discovery");
     shouldDiscover = YES;
     [_mdnsMan searchForHosts];
-    for (TemporaryHost* host in _hostQueue) {
-        [_opQueue addOperation:[self createWorkerForHost:host]];
+    
+    @synchronized (_hostQueue) {
+        for (TemporaryHost* host in _hostQueue) {
+            if (![_pausedHosts containsObject:host]) {
+                [_opQueue addOperation:[self createWorkerForHost:host]];
+            }
+        }
     }
 }
 
@@ -130,39 +279,64 @@
         if (host.localAddress != nil) {
             existingHost.localAddress = host.localAddress;
         }
+        if (host.ipv6Address != nil) {
+            existingHost.ipv6Address = host.ipv6Address;
+        }
         if (host.externalAddress != nil) {
             existingHost.externalAddress = host.externalAddress;
         }
         existingHost.activeAddress = host.activeAddress;
-        existingHost.online = host.online;
+        existingHost.state = host.state;
         return NO;
     }
     else {
-        // If we were added without an explicit address,
-        // populate it from our other available addresses
-        if (host.address == nil) {
-            if (host.externalAddress != nil) {
-                host.address = host.externalAddress;
+        @synchronized (_hostQueue) {
+            [_hostQueue addObject:host];
+            if (shouldDiscover) {
+                [_opQueue addOperation:[self createWorkerForHost:host]];
             }
-            else {
-                host.address = host.localAddress;
-            }
-        }
-        [_hostQueue addObject:host];
-        if (shouldDiscover) {
-            [_opQueue addOperation:[self createWorkerForHost:host]];
         }
         return YES;
     }
 }
 
 - (void) removeHostFromDiscovery:(TemporaryHost *)host {
-    for (DiscoveryWorker* worker in [_opQueue operations]) {
-        if ([worker getHost] == host) {
-            [worker cancel];
+    @synchronized (_hostQueue) {
+        for (DiscoveryWorker* worker in [_opQueue operations]) {
+            if ([worker getHost] == host) {
+                [worker cancel];
+            }
+        }
+        
+        [_hostQueue removeObject:host];
+        [_pausedHosts removeObject:host];
+    }
+}
+
+- (void) pauseDiscoveryForHost:(TemporaryHost *)host {
+    @synchronized (_hostQueue) {
+        // Stop any worker for the host
+        for (DiscoveryWorker* worker in [_opQueue operations]) {
+            if ([worker getHost] == host) {
+                [worker cancel];
+            }
+        }
+        
+        // Add it to the paused hosts list
+        [_pausedHosts addObject:host];
+    }
+}
+
+- (void) resumeDiscoveryForHost:(TemporaryHost *)host {
+    @synchronized (_hostQueue) {
+        // Remove it from the paused hosts list
+        [_pausedHosts removeObject:host];
+        
+        // Start discovery again
+        if (shouldDiscover) {
+            [_opQueue addOperation:[self createWorkerForHost:host]];
         }
     }
-    [_hostQueue removeObject:host];
 }
 
 // Override from MDNSCallback - called in a worker thread
@@ -174,16 +348,20 @@
     [worker discoverHost];
     if ([self addHostToDiscovery:host]) {
         Log(LOG_I, @"Found new host through MDNS: %@:", host.name);
-        [self->_callback updateAllHosts:self->_hostQueue];
+        @synchronized (_hostQueue) {
+            [_callback updateAllHosts:_hostQueue];
+        }
     } else {
         Log(LOG_D, @"Found existing host through MDNS: %@", host.name);
     }
 }
 
 - (TemporaryHost*) getHostInDiscovery:(NSString*)uuidString {
-    for (TemporaryHost* discoveredHost in _hostQueue) {
-        if (discoveredHost.uuid.length > 0 && [discoveredHost.uuid isEqualToString:uuidString]) {
-            return discoveredHost;
+    @synchronized (_hostQueue) {
+        for (TemporaryHost* discoveredHost in _hostQueue) {
+            if (discoveredHost.uuid.length > 0 && [discoveredHost.uuid isEqualToString:uuidString]) {
+                return discoveredHost;
+            }
         }
     }
     return nil;

Limelight/Network/DiscoveryWorker.m

@@ -52,6 +52,9 @@ static const float POLL_RATE = 2.0f; // Poll every 2 seconds
     if (_host.externalAddress != nil) {
         [array addObject:_host.externalAddress];
     }
+    if (_host.ipv6Address != nil) {
+        [array addObject:_host.ipv6Address];
+    }
     
     // Remove duplicate addresses from the list.
     // This is done using an array rather than a set
@@ -86,8 +89,9 @@ static const float POLL_RATE = 2.0f; // Poll every 2 seconds
     
     Log(LOG_D, @"%@ has %d unique addresses", _host.name, [addresses count]);
     
-    // Give the PC 2 tries to respond before declaring it offline
-    for (int i = 0; i < 2; i++) {
+    // Give the PC 2 tries to respond before declaring it offline if we've seen it before.
+    // If this is an unknown PC, update the status after 1 attempt to get the UI refreshed quickly.
+    for (int i = 0; i < (_host.state == StateUnknown ? 1 : 2); i++) {
         for (NSString *address in addresses) {
             if (self.cancelled) {
                 // Get out without updating the status because
@@ -115,9 +119,9 @@ static const float POLL_RATE = 2.0f; // Poll every 2 seconds
         }
     }
 
-    _host.online = receivedResponse;
+    _host.state = receivedResponse ? StateOnline : StateOffline;
     if (receivedResponse) {
-        Log(LOG_D, @"Received response from: %@\n{\n\t address:%@ \n\t localAddress:%@ \n\t externalAddress:%@ \n\t uuid:%@ \n\t mac:%@ \n\t pairState:%d \n\t online:%d \n\t activeAddress:%@ \n}", _host.name, _host.address, _host.localAddress, _host.externalAddress, _host.uuid, _host.mac, _host.pairState, _host.online, _host.activeAddress);
+        Log(LOG_D, @"Received response from: %@\n{\n\t address:%@ \n\t localAddress:%@ \n\t externalAddress:%@ \n\t ipv6Address:%@ \n\t uuid:%@ \n\t mac:%@ \n\t pairState:%d \n\t online:%d \n\t activeAddress:%@ \n}", _host.name, _host.address, _host.localAddress, _host.externalAddress, _host.ipv6Address, _host.uuid, _host.mac, _host.pairState, _host.state, _host.activeAddress);
     }
 }
 

Limelight/Network/HttpManager.m

@@ -14,6 +14,8 @@
 #include <libxml2/libxml/xmlreader.h>
 #include <string.h>
 
+#include <Limelight.h>
+
 #define SHORT_TIMEOUT_SEC 2
 #define NORMAL_TIMEOUT_SEC 5
 #define LONG_TIMEOUT_SEC 60
@@ -129,6 +131,10 @@ static const NSString* HTTPS_PORT = @"47984";
             [self executeRequestSynchronously:request];
         }
     }
+    else if (_error && request.response) {
+        request.response.statusCode = [_error code];
+        request.response.statusMessage = [_error localizedDescription];
+    }
 }
 
 - (NSURLRequest*) createRequestFromString:(NSString*) urlString timeout:(int)timeout {
@@ -198,9 +204,10 @@ static const NSString* HTTPS_PORT = @"47984";
 
 - (NSURLRequest*) newLaunchRequest:(StreamConfiguration*)config {
     // Using an FPS value over 60 causes SOPS to default to 720p60,
-    // so force it to 60 when starting. This won't impact our ability
-    // to get > 60 FPS while actually streaming though.
-    int fps = config.frameRate > 60 ? 60 : config.frameRate;
+    // so force it to 0 to ensure the correct resolution is set. We
+    // used to use 60 here but that locked the frame rate to 60 FPS
+    // on GFE 3.20.3.
+    int fps = config.frameRate > 60 ? 0 : config.frameRate;
     
     NSString* urlString = [NSString stringWithFormat:@"%@/launch?uniqueid=%@&appid=%@&mode=%dx%dx%d&additionalStates=1&sops=%d&rikey=%@&rikeyid=%d%@&localAudioPlayMode=%d&surroundAudioInfo=%d&remoteControllersBitmap=%d&gcmap=%d",
                            _baseHTTPSURL, _uniqueId,
@@ -210,7 +217,7 @@ static const NSString* HTTPS_PORT = @"47984";
                            [Utils bytesToHex:config.riKey], config.riKeyId,
                            config.enableHdr ? @"&hdrMode=1&clientHdrCapVersion=0&clientHdrCapSupportedFlagsInUint32=0&clientHdrCapMetaDataId=NV_STATIC_METADATA_TYPE_1&clientHdrCapDisplayData=0x0x0x0x0x0x0x0x0x0x0": @"",
                            config.playAudioOnPC ? 1 : 0,
-                           (config.audioChannelMask << 16) | config.audioChannelCount,
+                           SURROUNDAUDIOINFO_FROM_AUDIO_CONFIGURATION(config.audioConfiguration),
                            config.gamepadMask, config.gamepadMask];
     Log(LOG_I, @"Requesting: %@", urlString);
     // This blocks while the app is launching
@@ -221,7 +228,7 @@ static const NSString* HTTPS_PORT = @"47984";
     NSString* urlString = [NSString stringWithFormat:@"%@/resume?uniqueid=%@&rikey=%@&rikeyid=%d&surroundAudioInfo=%d",
                            _baseHTTPSURL, _uniqueId,
                            [Utils bytesToHex:config.riKey], config.riKeyId,
-                           (config.audioChannelMask << 16) | config.audioChannelCount];
+                           SURROUNDAUDIOINFO_FROM_AUDIO_CONFIGURATION(config.audioConfiguration)];
     Log(LOG_I, @"Requesting: %@", urlString);
     // This blocks while the app is resuming
     return [self createRequestFromString:urlString timeout:LONG_TIMEOUT_SEC];

Limelight/Network/HttpResponse.m

@@ -55,21 +55,26 @@
 
     xmlChar* statusStr = xmlGetProp(node, (const xmlChar*)[TAG_STATUS_CODE UTF8String]);
     if (statusStr != NULL) {
-        int status = [[NSString stringWithUTF8String:(const char*)statusStr] intValue];
+        int status = (int)[[NSString stringWithUTF8String:(const char*)statusStr] longLongValue];
         xmlFree(statusStr);
         self.statusCode = status;
     }
     
     xmlChar* statusMsgXml = xmlGetProp(node, (const xmlChar*)[TAG_STATUS_MESSAGE UTF8String]);
-    NSString* statusMsg;
     if (statusMsgXml != NULL) {
-        statusMsg = [NSString stringWithUTF8String:(const char*)statusMsgXml];
+        self.statusMessage = [NSString stringWithUTF8String:(const char*)statusMsgXml];
         xmlFree(statusMsgXml);
     }
     else {
-        statusMsg = @"Server Error";
+        self.statusMessage = @"Server Error";
+    }
+    
+    if (self.statusCode == -1 && [self.statusMessage isEqualToString:@"Invalid"]) {
+        // Special case handling an audio capture error which GFE doesn't
+        // provide any useful status message for.
+        self.statusCode = 418;
+        self.statusMessage = @"Missing audio capture device. Reinstalling GeForce Experience should resolve this error.";
     }
-    self.statusMessage = statusMsg;
 
     node = node->children;
     

Limelight/Network/MDNSManager.h

@@ -21,6 +21,7 @@
 - (id) initWithCallback:(id<MDNSCallback>) callback;
 - (void) searchForHosts;
 - (void) stopSearching;
+- (void) forgetHosts;
 
 @end
 

Limelight/Network/MDNSManager.m

@@ -63,26 +63,166 @@ static NSString* NV_SERVICE_TYPE = @"_nvstream._tcp";
     [mDNSBrowser stop];
 }
 
+- (void) forgetHosts {
+    [services removeAllObjects];
+}
+
++ (NSString*)sockAddrToString:(NSData*)addrData {
+    char addrStr[INET6_ADDRSTRLEN];
+    struct sockaddr* addr = (struct sockaddr*)[addrData bytes];
+    if (addr->sa_family == AF_INET) {
+        inet_ntop(addr->sa_family, &((struct sockaddr_in*)addr)->sin_addr, addrStr, sizeof(addrStr));
+    }
+    else {
+        struct sockaddr_in6* sin6 = (struct sockaddr_in6*)addr;
+        inet_ntop(addr->sa_family, &sin6->sin6_addr, addrStr, sizeof(addrStr));
+        if (sin6->sin6_scope_id != 0) {
+            // Link-local addresses with scope IDs are special
+            return [NSString stringWithFormat: @"%s%%%u", addrStr, sin6->sin6_scope_id];
+        }
+    }
+    return [NSString stringWithFormat: @"%s", addrStr];
+}
+
++ (BOOL)isAddress:(uint8_t*)address inSubnet:(uint8_t*)subnet netmask:(int)bits {
+    for (int i = 0; i < bits; i++) {
+        uint8_t mask = 1 << (i % 8);
+        if ((address[i / 8] & mask) != (subnet[i / 8] & mask)) {
+            return NO;
+        }
+    }
+    return YES;
+}
+
++ (BOOL)isLocalIpv6Address:(NSData*)addrData {
+    struct sockaddr_in6* sin6 = (struct sockaddr_in6*)[addrData bytes];
+    if (sin6->sin6_family != AF_INET6) {
+        return NO;
+    }
+    
+    uint8_t* addrBytes = sin6->sin6_addr.s6_addr;
+    uint8_t prefix[2];
+    
+    // fe80::/10
+    prefix[0] = 0xfe;
+    prefix[1] = 0x80;
+    if ([MDNSManager isAddress:addrBytes inSubnet:prefix netmask:10]) {
+        // Link-local
+        return YES;
+    }
+    
+    // fec0::/10
+    prefix[0] = 0xfe;
+    prefix[1] = 0xc0;
+    if ([MDNSManager isAddress:addrBytes inSubnet:prefix netmask:10]) {
+        // Site local
+        return YES;
+    }
+    
+    // fc00::/7
+    prefix[0] = 0xfc;
+    prefix[1] = 0x00;
+    if ([MDNSManager isAddress:addrBytes inSubnet:prefix netmask:7]) {
+        // ULA
+        return YES;
+    }
+    
+    return NO;
+}
+
++ (NSString*)getBestIpv6Address:(NSArray<NSData*>*)addresses {
+    for (NSData* addrData in addresses) {
+        struct sockaddr_in6* sin6 = (struct sockaddr_in6*)[addrData bytes];
+        if (sin6->sin6_family != AF_INET6) {
+            continue;
+        }
+        
+        if ([MDNSManager isLocalIpv6Address:addrData]) {
+            // Skip non-global addresses
+            continue;
+        }
+        
+        uint8_t* addrBytes = sin6->sin6_addr.s6_addr;
+        uint8_t prefix[2];
+        
+        // 2002::/16
+        prefix[0] = 0x20;
+        prefix[1] = 0x02;
+        if ([MDNSManager isAddress:addrBytes inSubnet:prefix netmask:16]) {
+            Log(LOG_I, @"Ignoring 6to4 address: %@", [MDNSManager sockAddrToString:addrData]);
+            continue;
+        }
+        
+        // 2001::/32
+        prefix[0] = 0x20;
+        prefix[1] = 0x01;
+        if ([MDNSManager isAddress:addrBytes inSubnet:prefix netmask:32]) {
+            Log(LOG_I, @"Ignoring Teredo address: %@", [MDNSManager sockAddrToString:addrData]);
+            continue;
+        }
+        
+        return [MDNSManager sockAddrToString:addrData];
+    }
+    
+    return nil;
+}
+
 - (void)netServiceDidResolveAddress:(NSNetService *)service {
     dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
-        Log(LOG_D, @"Resolved address: %@ -> %@", service, service.hostName);
+        NSArray<NSData*>* addresses = [service addresses];
+        
+        for (NSData* addrData in addresses) {
+            Log(LOG_I, @"Resolved address: %@ -> %@", [service hostName], [MDNSManager sockAddrToString: addrData]);
+        }
+        
         TemporaryHost* host = [[TemporaryHost alloc] init];
         
-        // Since we discovered this host over mDNS, we know we're on the same network
-        // as the PC and we can use our current WAN address as a likely candidate
-        // for our PC's external address.
-        struct in_addr wanAddr;
-        int err = LiFindExternalAddressIP4("stun.moonlight-stream.org", 3478, &wanAddr.s_addr);
-        if (err == 0) {
-            char addrStr[INET_ADDRSTRLEN];
-            inet_ntop(AF_INET, &wanAddr, addrStr, sizeof(addrStr));
-            host.externalAddress = [NSString stringWithFormat: @"%s", addrStr];
-        }
-        else {
-            Log(LOG_E, @"STUN failed to get WAN address: %d", err);
+        // First, look for an IPv4 record for the local address
+        for (NSData* addrData in addresses) {
+            struct sockaddr_in* sin = (struct sockaddr_in*)[addrData bytes];
+            if (sin->sin_family != AF_INET) {
+                continue;
+            }
+            
+            // Don't send a STUN request if we're connected to a VPN. We'll likely get the VPN
+            // gateway's external address rather than the external address of the LAN.
+            if (![Utils isActiveNetworkVPN]) {
+                // Since we discovered this host over IPv4 mDNS, we know we're on the same network
+                // as the PC and we can use our current WAN address as a likely candidate
+                // for our PC's external address.
+                struct in_addr wanAddr;
+                int err = LiFindExternalAddressIP4("stun.moonlight-stream.org", 3478, &wanAddr.s_addr);
+                if (err == 0) {
+                    char addrStr[INET_ADDRSTRLEN];
+                    inet_ntop(AF_INET, &wanAddr, addrStr, sizeof(addrStr));
+                    host.externalAddress = [NSString stringWithFormat: @"%s", addrStr];
+                    Log(LOG_I, @"External IPv4 address (STUN): %@ -> %@", [service hostName], host.externalAddress);
+                }
+                else {
+                    Log(LOG_E, @"STUN failed to get WAN address: %d", err);
+                }
+            }
+            
+            host.localAddress = [MDNSManager sockAddrToString:addrData];
+            Log(LOG_I, @"Local address chosen: %@ -> %@", [service hostName], host.localAddress);
+            break;
         }
         
-        host.activeAddress = host.localAddress = service.hostName;
+        if (host.localAddress == nil) {
+            // If we didn't find an IPv4 record, look for a local IPv6 record
+            for (NSData* addrData in addresses) {
+                if ([MDNSManager isLocalIpv6Address:addrData]) {
+                    host.localAddress = [MDNSManager sockAddrToString:addrData];
+                    Log(LOG_I, @"Local address chosen: %@ -> %@", [service hostName], host.localAddress);
+                    break;
+                }
+            }
+        }
+        
+        host.ipv6Address = [MDNSManager getBestIpv6Address:addresses];
+        Log(LOG_I, @"IPv6 address chosen: %@ -> %@", [service hostName], host.ipv6Address);
+        
+        host.activeAddress = host.localAddress;
         host.name = service.hostName;
         [self.callback updateHost:host];
     });

Limelight/Network/PairManager.h

@@ -10,7 +10,7 @@
 
 @protocol PairCallback <NSObject>
 
-- (void) showPIN:(NSString*)PIN;
+- (void) startPairing:(NSString*)PIN;
 - (void) pairSuccessful:(NSData*)serverCert;
 - (void) pairFailed:(NSString*)message;
 - (void) alreadyPaired;
@@ -19,8 +19,4 @@
 
 @interface PairManager : NSOperation
 - (id) initWithManager:(HttpManager*)httpManager clientCert:(NSData*)clientCert callback:(id<PairCallback>)callback;
-- (NSString*) generatePIN;
-- (NSData*) saltPIN:(NSString*)PIN;
-- (void) initiatePair:(int)serverMajorVersion;
-
 @end

Limelight/Network/PairManager.m

@@ -30,53 +30,82 @@
 }
 
 - (void) main {
+    // We have to call startPairing before calling any other _callback functions
+    NSString* PIN = [self generatePIN];
+    [_callback startPairing:PIN];
+    
     ServerInfoResponse* serverInfoResp = [[ServerInfoResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:serverInfoResp withUrlRequest:[_httpManager newServerInfoRequest:false]
                                                fallbackError:401 fallbackRequest:[_httpManager newHttpServerInfoRequest]]];
-    if (serverInfoResp == nil) {
-        [_callback pairFailed:@"Unable to connect to PC"];
-        return;
-    }
     if ([serverInfoResp isStatusOk]) {
         if ([[serverInfoResp getStringTag:@"state"] hasSuffix:@"_SERVER_BUSY"]) {
-            [_callback pairFailed:@"You must stop streaming before attempting to pair."];
+            [_callback pairFailed:@"You cannot pair while a previous session is still running on the host PC. Quit any running games or reboot the host PC, then try pairing again."];
         } else if (![[serverInfoResp getStringTag:@"PairStatus"] isEqual:@"1"]) {
             NSString* appversion = [serverInfoResp getStringTag:@"appversion"];
             if (appversion == nil) {
                 [_callback pairFailed:@"Missing XML element"];
                 return;
             }            
-            [self initiatePair: [[appversion substringToIndex:1] intValue]];
+            [self initiatePairWithPin:PIN forServerMajorVersion:[[appversion substringToIndex:1] intValue]];
         } else {
             [_callback alreadyPaired];
         }
     }
+    else {
+        [_callback pairFailed:serverInfoResp.statusMessage];
+    }
 }
 
-// All codepaths must call pairFailed or pairSuccessful exactly once before returning!
-- (void) initiatePair:(int)serverMajorVersion {
+- (void) finishPairing:(UIBackgroundTaskIdentifier)bgId
+           forResponse:(HttpResponse*)resp
+     withFallbackError:(NSString*)errorMsg {
+    [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
+    
+    if (bgId != UIBackgroundTaskInvalid) {
+        [[UIApplication sharedApplication] endBackgroundTask:bgId];
+    }
+    
+    if (![resp isStatusOk]) {
+        // Use the response error if the request failed
+        errorMsg = resp.statusMessage;
+    }
+    
+    [_callback pairFailed:errorMsg];
+}
+
+- (void) finishPairing:(UIBackgroundTaskIdentifier)bgId withSuccess:(NSData*)derCertBytes {
+    if (bgId != UIBackgroundTaskInvalid) {
+        [[UIApplication sharedApplication] endBackgroundTask:bgId];
+    }
+    
+    [_callback pairSuccessful:derCertBytes];
+}
+
+// All codepaths must call finishPairing exactly once before returning!
+- (void) initiatePairWithPin:(NSString*)PIN forServerMajorVersion:(int)serverMajorVersion {
     Log(LOG_I, @"Pairing with generation %d server", serverMajorVersion);
     
-    NSString* PIN = [self generatePIN];
-    NSData* salt = [self saltPIN:PIN];
-    Log(LOG_I, @"PIN: %@, saltedPIN: %@", PIN, salt);
-    [_callback showPIN:PIN];
+    // Start a background task to help prevent the app from being killed
+    // while pairing is in progress.
+    UIBackgroundTaskIdentifier bgId = [[UIApplication sharedApplication] beginBackgroundTaskWithName:@"Pairing PC" expirationHandler:^{
+        Log(LOG_W, @"Background pairing time has expired!");
+    }];
+    
+    NSData* salt = [Utils randomBytes:16];
+    NSData* saltedPIN = [self concatData:salt with:[PIN dataUsingEncoding:NSUTF8StringEncoding]];
+
+    Log(LOG_I, @"PIN: %@, salt %@", PIN, salt);
     
     HttpResponse* pairResp = [[HttpResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:pairResp withUrlRequest:[_httpManager newPairRequest:salt clientCert:_clientCert]]];
     if (![self verifyResponseStatus:pairResp]) {
-        [_callback pairFailed:@"Pairing stage #1 failed"];
-        return;
-    }
-    NSInteger pairedStatus;
-    if (![pairResp getIntTag:@"paired" value:&pairedStatus] || !pairedStatus) {
-        [_callback pairFailed:@"Pairing was declined by the target."];
+        [self finishPairing:bgId forResponse:pairResp withFallbackError:@"Pairing was declined by the target."];
         return;
     }
     
     NSString* plainCert = [pairResp getStringTag:@"plaincert"];
     if ([plainCert length] == 0) {
-        [_callback pairFailed:@"Another pairing attempt is already in progress."];
+        [self finishPairing:bgId forResponse:pairResp withFallbackError:@"Another pairing attempt is already in progress."];
         return;
     }
     
@@ -90,11 +119,11 @@
     // Gen 7 servers use SHA256 to get the key
     int hashLength;
     if (serverMajorVersion >= 7) {
-        aesKey = [cryptoMan createAESKeyFromSaltSHA256:salt];
+        aesKey = [cryptoMan createAESKeyFromSaltSHA256:saltedPIN];
         hashLength = 32;
     }
     else {
-        aesKey = [cryptoMan createAESKeyFromSaltSHA1:salt];
+        aesKey = [cryptoMan createAESKeyFromSaltSHA1:saltedPIN];
         hashLength = 20;
     }
     
@@ -103,11 +132,8 @@
     
     HttpResponse* challengeResp = [[HttpResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:challengeResp withUrlRequest:[_httpManager newChallengeRequest:encryptedChallenge]]];
-    if (![self verifyResponseStatus:challengeResp] ||
-        ![challengeResp getIntTag:@"paired" value:&pairedStatus] ||
-        pairedStatus != 1) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Pairing stage #2 failed"];
+    if (![self verifyResponseStatus:challengeResp]) {
+        [self finishPairing:bgId forResponse:challengeResp withFallbackError:@"Pairing stage #2 failed"];
         return;
     }
     
@@ -126,15 +152,17 @@
     else {
         challengeRespHash = [cryptoMan SHA1HashData: challengeRespHashInput];
     }
-    NSData* challengeRespEncrypted = [cryptoMan aesEncrypt:challengeRespHash withKey:aesKey];
+    
+    assert([challengeRespHash length] <= 32);
+    NSMutableData* paddedHash = [NSMutableData dataWithData:challengeRespHash];
+    [paddedHash setLength:32];
+    
+    NSData* challengeRespEncrypted = [cryptoMan aesEncrypt:paddedHash withKey:aesKey];
     
     HttpResponse* secretResp = [[HttpResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:secretResp withUrlRequest:[_httpManager newChallengeRespRequest:challengeRespEncrypted]]];
-    if (![self verifyResponseStatus:secretResp] ||
-        ![secretResp getIntTag:@"paired" value:&pairedStatus] ||
-        pairedStatus != 1) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Pairing stage #3 failed"];
+    if (![self verifyResponseStatus:secretResp]) {
+        [self finishPairing:bgId forResponse:secretResp withFallbackError:@"Pairing stage #3 failed"];
         return;
     }
     
@@ -143,8 +171,7 @@
     NSData* serverSignature = [serverSecretResp subdataWithRange:NSMakeRange(16, 256)];
     
     if (![cryptoMan verifySignature:serverSecret withSignature:serverSignature andCert:[Utils hexToBytes:plainCert]]) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Server certificate invalid"];
+        [self finishPairing:bgId forResponse:secretResp withFallbackError:@"Server certificate invalid"];
         return;
     }
     
@@ -157,46 +184,40 @@
         serverChallengeRespHash = [cryptoMan SHA1HashData: serverChallengeRespHashInput];
     }
     if (![serverChallengeRespHash isEqual:serverResponse]) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Incorrect PIN"];
+        [self finishPairing:bgId forResponse:secretResp withFallbackError:@"Incorrect PIN"];
         return;
     }
     
     NSData* clientPairingSecret = [self concatData:clientSecret with:[cryptoMan signData:clientSecret withKey:[CryptoManager readKeyFromFile]]];
     HttpResponse* clientSecretResp = [[HttpResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:clientSecretResp withUrlRequest:[_httpManager newClientSecretRespRequest:[Utils bytesToHex:clientPairingSecret]]]];
-    if (![self verifyResponseStatus:clientSecretResp] ||
-        ![clientSecretResp getIntTag:@"paired" value:&pairedStatus] ||
-        pairedStatus != 1) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Pairing stage #4 failed"];
+    if (![self verifyResponseStatus:clientSecretResp]) {
+        [self finishPairing:bgId forResponse:clientSecretResp withFallbackError:@"Pairing stage #4 failed"];
         return;
     }
     
     HttpResponse* clientPairChallengeResp = [[HttpResponse alloc] init];
     [_httpManager executeRequestSynchronously:[HttpRequest requestForResponse:clientPairChallengeResp withUrlRequest:[_httpManager newPairChallenge]]];
-    if (![self verifyResponseStatus:clientPairChallengeResp] ||
-        ![clientPairChallengeResp getIntTag:@"paired" value:&pairedStatus] ||
-        pairedStatus != 1) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Pairing stage #5 failed"];
+    if (![self verifyResponseStatus:clientPairChallengeResp]) {
+        [self finishPairing:bgId forResponse:clientPairChallengeResp withFallbackError:@"Pairing stage #5 failed"];
         return;
     }
     
-    [_callback pairSuccessful: derCertBytes];
+    [self finishPairing:bgId withSuccess:derCertBytes];
 }
 
+// Caller calls finishPairing for us on failure
 - (BOOL) verifyResponseStatus:(HttpResponse*)resp {
-    if (resp == nil) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:@"Network error occured."];
-        return false;
-    } else if (![resp isStatusOk]) {
-        [_httpManager executeRequestSynchronously:[HttpRequest requestWithUrlRequest:[_httpManager newUnpairRequest]]];
-        [_callback pairFailed:resp.statusMessage];
+    if (![resp isStatusOk]) {
         return false;
     } else {
-        return true;
+        NSInteger pairedStatus;
+        
+        if (![resp getIntTag:@"paired" value:&pairedStatus]) {
+            return false;
+        }
+        
+        return pairedStatus == 1;
     }
 }
 
@@ -208,16 +229,9 @@
 
 - (NSString*) generatePIN {
     NSString* PIN = [NSString stringWithFormat:@"%d%d%d%d",
-                     arc4random() % 10, arc4random() % 10,
-                     arc4random() % 10, arc4random() % 10];
+                     arc4random_uniform(10), arc4random_uniform(10),
+                     arc4random_uniform(10), arc4random_uniform(10)];
     return PIN;
 }
 
-- (NSData*) saltPIN:(NSString*)PIN {
-    NSMutableData* saltedPIN = [[NSMutableData alloc] initWithCapacity:20];
-    [saltedPIN appendData:[Utils randomBytes:16]];
-    [saltedPIN appendBytes:[PIN UTF8String] length:4];
-    return saltedPIN;
-}
-
 @end

Limelight/Network/ServerInfoResponse.m

@@ -19,11 +19,16 @@
 
 - (void) populateHost:(TemporaryHost*)host {
     host.name = [[self getStringTag:TAG_HOSTNAME] trim];
-    host.localAddress = [[self getStringTag:TAG_LOCAL_IP] trim];
     host.uuid = [[self getStringTag:TAG_UNIQUE_ID] trim];
     host.mac = [[self getStringTag:TAG_MAC_ADDRESS] trim];
     host.currentGame = [[self getStringTag:TAG_CURRENT_GAME] trim];
     
+    // We might get an IPv4 loopback address if we're using GS IPv6 Forwarder
+    NSString *lanAddr = [[self getStringTag:TAG_LOCAL_IP] trim];
+    if (![lanAddr hasPrefix:@"127."]) {
+        host.localAddress = lanAddr;
+    }
+    
     // Modern GFE versions don't actually give us a WAN address anymore
     // so we leave the one that we populated from mDNS discovery via STUN.
     NSString *wanAddr = [[self getStringTag:TAG_EXTERNAL_IP] trim];

Limelight/Network/WakeOnLanManager.m

@@ -16,8 +16,12 @@
 
 @implementation WakeOnLanManager
 
-static const int numPorts = 5;
-static const int ports[numPorts] = {7, 9, 47998, 47999, 48000};
+static const int numPorts = 7;
+static const int ports[numPorts] = {
+    9, // Standard WOL port (privileged port)
+    47998, 47999, 48000, 48002, 48010, // Ports opened by GFE
+    47009, // Port opened by Moonlight Internet Hosting Tool for WoL (non-privileged port)
+};
 
 + (void) populateAddress:(struct sockaddr_storage*)addr withPort:(unsigned short)port {
     if (addr->ss_family == AF_INET) {
@@ -33,7 +37,7 @@ static const int ports[numPorts] = {7, 9, 47998, 47999, 48000};
 + (void) wakeHost:(TemporaryHost*)host {
     NSData* wolPayload = [WakeOnLanManager createPayload:host];
     
-    for (int i = 0; i < 4; i++) {
+    for (int i = 0; i < 5; i++) {
         const char* address;
         struct addrinfo hints, *res, *curr;
         
@@ -44,7 +48,9 @@ static const int ports[numPorts] = {7, 9, 47998, 47999, 48000};
             address = [host.externalAddress UTF8String];
         } else if (i == 2 && host.address != nil) {
             address = [host.address UTF8String];
-        } else if (i == 3) {
+        } else if (i == 3 && host.ipv6Address != nil) {
+            address = [host.ipv6Address UTF8String];
+        } else if (i == 4) {
             address = "255.255.255.255";
         } else {
             // Requested address wasn't present

Limelight/OSPortabilityDefs.h

@@ -1,24 +0,0 @@
-//
-//  OSPortabilityDefs.h
-//  Moonlight
-//
-//  Created by Cameron Gutman on 3/26/18.
-//  Copyright © 2018 Moonlight Stream. All rights reserved.
-//
-
-#ifndef OSPortabilityDefs_h
-#define OSPortabilityDefs_h
-
-#if TARGET_OS_IPHONE
-#define OSImage UIImage
-#define OSColor UIColor
-#define OSView UIView
-#define OSApplication UIApplication
-#else
-#define OSImage NSImage
-#define OSColor NSColor
-#define OSView NSView
-#define OSApplication NSApplication
-#endif
-
-#endif /* OSPortabilityDefs_h */

Limelight/Stream/Connection.h

@@ -9,22 +9,22 @@
 #import "VideoDecoderRenderer.h"
 #import "StreamConfiguration.h"
 
-@protocol ConnectionCallbacks <NSObject>
+#define CONN_TEST_SERVER "ios.conntest.moonlight-stream.org"
 
-- (void) connectionStarted;
-- (void) connectionTerminated:(long)errorCode;
-- (void) stageStarting:(const char*)stageName;
-- (void) stageComplete:(const char*)stageName;
-- (void) stageFailed:(const char*)stageName withError:(long)errorCode;
-- (void) launchFailed:(NSString*)message;
-- (void) rumble:(unsigned short)controllerNumber lowFreqMotor:(unsigned short)lowFreqMotor highFreqMotor:(unsigned short)highFreqMotor;
-
-@end
+typedef struct {
+    CFTimeInterval startTime;
+    CFTimeInterval endTime;
+    int totalFrames;
+    int receivedFrames;
+    int networkDroppedFrames;
+} video_stats_t;
 
 @interface Connection : NSOperation <NSStreamDelegate>
 
 -(id) initWithConfig:(StreamConfiguration*)config renderer:(VideoDecoderRenderer*)myRenderer connectionCallbacks:(id<ConnectionCallbacks>)callbacks;
 -(void) terminate;
 -(void) main;
+-(BOOL) getVideoStats:(video_stats_t*)stats;
+-(NSString*) getActiveCodecName;
 
 @end

Limelight/Stream/Connection.m

@@ -7,11 +7,13 @@
 //
 
 #import "Connection.h"
+#import "Utils.h"
 
-#import <AudioUnit/AudioUnit.h>
-#import <AVFoundation/AVFoundation.h>
 #import <VideoToolbox/VideoToolbox.h>
 
+#define SDL_MAIN_HANDLED
+#import <SDL.h>
+
 #include "Limelight.h"
 #include "opus_multistream.h"
 
@@ -24,36 +26,70 @@
     char _hostString[256];
     char _appVersionString[32];
     char _gfeVersionString[32];
+    char _rtspSessionUrl[128];
 }
 
 static NSLock* initLock;
 static OpusMSDecoder* opusDecoder;
 static id<ConnectionCallbacks> _callbacks;
+static int lastFrameNumber;
+static int activeVideoFormat;
+static video_stats_t currentVideoStats;
+static video_stats_t lastVideoStats;
+static NSLock* videoStatsLock;
 
-#define OUTPUT_BUS 0
+static SDL_AudioDeviceID audioDevice;
+static OPUS_MULTISTREAM_CONFIGURATION audioConfig;
+static void* audioBuffer;
+static int audioFrameSize;
 
-#define MAX_CHANNEL_COUNT 2
-#define FRAME_SIZE 240
-
-#define CIRCULAR_BUFFER_SIZE 32
-
-static int audioBufferWriteIndex;
-static int audioBufferReadIndex;
-static int activeChannelCount;
-static short audioCircularBuffer[CIRCULAR_BUFFER_SIZE][FRAME_SIZE * MAX_CHANNEL_COUNT];
-
-#define AUDIO_QUEUE_BUFFERS 4
-
-static AudioQueueRef audioQueue;
-static AudioQueueBufferRef audioBuffers[AUDIO_QUEUE_BUFFERS];
 static VideoDecoderRenderer* renderer;
 
 int DrDecoderSetup(int videoFormat, int width, int height, int redrawRate, void* context, int drFlags)
 {
-    [renderer setupWithVideoFormat:videoFormat];
+    [renderer setupWithVideoFormat:videoFormat refreshRate:redrawRate];
+    lastFrameNumber = 0;
+    activeVideoFormat = videoFormat;
+    memset(&currentVideoStats, 0, sizeof(currentVideoStats));
+    memset(&lastVideoStats, 0, sizeof(lastVideoStats));
     return 0;
 }
 
+void DrCleanup(void)
+{
+    [renderer cleanup];
+}
+
+-(BOOL) getVideoStats:(video_stats_t*)stats
+{
+    // We return lastVideoStats because it is a complete 1 second window
+    [videoStatsLock lock];
+    if (lastVideoStats.endTime != 0) {
+        memcpy(stats, &lastVideoStats, sizeof(*stats));
+        [videoStatsLock unlock];
+        return YES;
+    }
+    
+    // No stats yet
+    [videoStatsLock unlock];
+    return NO;
+}
+
+-(NSString*) getActiveCodecName
+{
+    switch (activeVideoFormat)
+    {
+        case VIDEO_FORMAT_H264:
+            return @"H.264";
+        case VIDEO_FORMAT_H265:
+            return @"HEVC";
+        case VIDEO_FORMAT_H265_MAIN10:
+            return @"HEVC Main 10";
+        default:
+            return @"UNKNOWN";
+    }
+}
+
 int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
 {
     int offset = 0;
@@ -63,12 +99,43 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
         // A frame was lost due to OOM condition
         return DR_NEED_IDR;
     }
+    
+    CFTimeInterval now = CACurrentMediaTime();
+    if (!lastFrameNumber) {
+        currentVideoStats.startTime = now;
+        lastFrameNumber = decodeUnit->frameNumber;
+    }
+    else {
+        // Flip stats roughly every second
+        if (now - currentVideoStats.startTime >= 1.0f) {
+            currentVideoStats.endTime = now;
+            
+            [videoStatsLock lock];
+            lastVideoStats = currentVideoStats;
+            [videoStatsLock unlock];
+            
+            memset(&currentVideoStats, 0, sizeof(currentVideoStats));
+            currentVideoStats.startTime = now;
+        }
+        
+        // Any frame number greater than m_LastFrameNumber + 1 represents a dropped frame
+        currentVideoStats.networkDroppedFrames += decodeUnit->frameNumber - (lastFrameNumber + 1);
+        currentVideoStats.totalFrames += decodeUnit->frameNumber - (lastFrameNumber + 1);
+        lastFrameNumber = decodeUnit->frameNumber;
+    }
+    
+    currentVideoStats.receivedFrames++;
+    currentVideoStats.totalFrames++;
 
     PLENTRY entry = decodeUnit->bufferList;
     while (entry != NULL) {
         // Submit parameter set NALUs directly since no copy is required by the decoder
         if (entry->bufferType != BUFFER_TYPE_PICDATA) {
-            ret = [renderer submitDecodeBuffer:(unsigned char*)entry->data length:entry->length bufferType:entry->bufferType];
+            ret = [renderer submitDecodeBuffer:(unsigned char*)entry->data
+                                        length:entry->length
+                                    bufferType:entry->bufferType
+                                     frameType:decodeUnit->frameType
+                                           pts:decodeUnit->presentationTimeMs];
             if (ret != DR_OK) {
                 free(data);
                 return ret;
@@ -83,77 +150,56 @@ int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit)
     }
 
     // This function will take our picture data buffer
-    return [renderer submitDecodeBuffer:data length:offset bufferType:BUFFER_TYPE_PICDATA];
+    return [renderer submitDecodeBuffer:data
+                                 length:offset
+                             bufferType:BUFFER_TYPE_PICDATA
+                              frameType:decodeUnit->frameType
+                                    pts:decodeUnit->presentationTimeMs];
 }
 
 int ArInit(int audioConfiguration, POPUS_MULTISTREAM_CONFIGURATION opusConfig, void* context, int flags)
 {
     int err;
+    SDL_AudioSpec want, have;
     
-    // Clear the circular buffer
-    audioBufferWriteIndex = audioBufferReadIndex = 0;
+    if (SDL_InitSubSystem(SDL_INIT_AUDIO) < 0) {
+        Log(LOG_E, @"Failed to initialize audio subsystem: %s\n", SDL_GetError());
+        return -1;
+    }
+        
+    SDL_zero(want);
+    want.freq = opusConfig->sampleRate;
+    want.format = AUDIO_S16;
+    want.channels = opusConfig->channelCount;
 
-    // We only support stereo for now.
-    // TODO: Ensure AudioToolbox's channel mapping matches Opus's
-    // and correct if neccessary.
-    assert(audioConfiguration == AUDIO_CONFIGURATION_STEREO);
+    // This is supposed to be a power of 2, but our
+    // frames contain a non-power of 2 number of samples,
+    // so the slop would require buffering another full frame.
+    // Specifying non-Po2 seems to work for our supported platforms.
+    want.samples = opusConfig->samplesPerFrame;
 
-    activeChannelCount = opusConfig->channelCount;
+    audioDevice = SDL_OpenAudioDevice(NULL, 0, &want, &have, 0);
+    if (audioDevice == 0) {
+        Log(LOG_E, @"Failed to open audio device: %s\n", SDL_GetError());
+        SDL_QuitSubSystem(SDL_INIT_AUDIO);
+        return -1;
+    }
+    
+    audioConfig = *opusConfig;
+    audioFrameSize = opusConfig->samplesPerFrame * sizeof(short) * opusConfig->channelCount;
+    audioBuffer = SDL_malloc(audioFrameSize);
+    
     opusDecoder = opus_multistream_decoder_create(opusConfig->sampleRate,
                                                   opusConfig->channelCount,
                                                   opusConfig->streams,
                                                   opusConfig->coupledStreams,
                                                   opusConfig->mapping,
                                                   &err);
-
-#if TARGET_OS_IPHONE
-    // Configure the audio session for our app
-    NSError *audioSessionError = nil;
-    AVAudioSession* audioSession = [AVAudioSession sharedInstance];
-
-    [audioSession setPreferredSampleRate:opusConfig->sampleRate error:&audioSessionError];
-    [audioSession setCategory: AVAudioSessionCategoryPlayback error: &audioSessionError];
-    [audioSession setPreferredOutputNumberOfChannels:opusConfig->channelCount error:&audioSessionError];
-    [audioSession setPreferredIOBufferDuration:0.005 error:&audioSessionError];
-    [audioSession setActive: YES error: &audioSessionError];
-#endif
     
-    OSStatus status;
+    // Start playback
+    SDL_PauseAudioDevice(audioDevice, 0);
     
-    AudioStreamBasicDescription audioFormat = {0};
-    audioFormat.mSampleRate = opusConfig->sampleRate;
-    audioFormat.mBitsPerChannel = 16;
-    audioFormat.mFormatID = kAudioFormatLinearPCM;
-    audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked;
-    audioFormat.mChannelsPerFrame = opusConfig->channelCount;
-    audioFormat.mBytesPerFrame = audioFormat.mChannelsPerFrame * (audioFormat.mBitsPerChannel / 8);
-    audioFormat.mBytesPerPacket = audioFormat.mBytesPerFrame;
-    audioFormat.mFramesPerPacket = audioFormat.mBytesPerPacket / audioFormat.mBytesPerFrame;
-    audioFormat.mReserved = 0;
-
-    status = AudioQueueNewOutput(&audioFormat, FillOutputBuffer, nil, nil, nil, 0, &audioQueue);
-    if (status != noErr) {
-        Log(LOG_E, @"Error allocating output queue: %d\n", status);
-        return status;
-    }
-    
-    for (int i = 0; i < AUDIO_QUEUE_BUFFERS; i++) {
-        status = AudioQueueAllocateBuffer(audioQueue, audioFormat.mBytesPerFrame * FRAME_SIZE, &audioBuffers[i]);
-        if (status != noErr) {
-            Log(LOG_E, @"Error allocating output buffer: %d\n", status);
-            return status;
-        }
-        
-        FillOutputBuffer(nil, audioQueue, audioBuffers[i]);
-    }
-    
-    status = AudioQueueStart(audioQueue, nil);
-    if (status != noErr) {
-        Log(LOG_E, @"Error starting queue: %d\n", status);
-        return status;
-    }
-    
-    return status;
+    return 0;
 }
 
 void ArCleanup(void)
@@ -163,40 +209,43 @@ void ArCleanup(void)
         opusDecoder = NULL;
     }
     
-    // Stop before disposing to avoid massive delay inside
-    // AudioQueueDispose() (iOS bug?)
-    AudioQueueStop(audioQueue, true);
+    if (audioDevice != 0) {
+        SDL_CloseAudioDevice(audioDevice);
+        audioDevice = 0;
+    }
     
-    // Also frees buffers
-    AudioQueueDispose(audioQueue, true);
+    if (audioBuffer != NULL) {
+        SDL_free(audioBuffer);
+        audioBuffer = NULL;
+    }
     
-#if TARGET_OS_IPHONE
-    // Audio session is now inactive
-    AVAudioSession* audioSession = [AVAudioSession sharedInstance];
-    [audioSession setActive: YES error: nil];
-#endif
+    SDL_QuitSubSystem(SDL_INIT_AUDIO);
 }
 
 void ArDecodeAndPlaySample(char* sampleData, int sampleLength)
 {
     int decodeLen;
     
-    // Check if there is space for this sample in the buffer. Again, this can race
-    // but in the worst case, we'll not see the sample callback having consumed a sample.
-    if (((audioBufferWriteIndex + 1) % CIRCULAR_BUFFER_SIZE) == audioBufferReadIndex) {
+    // Don't queue if there's already more than 30 ms of audio data waiting
+    // in Moonlight's audio queue.
+    if (LiGetPendingAudioDuration() > 30) {
         return;
     }
     
     decodeLen = opus_multistream_decode(opusDecoder, (unsigned char *)sampleData, sampleLength,
-                                        audioCircularBuffer[audioBufferWriteIndex], FRAME_SIZE, 0);
+                                        (short*)audioBuffer, audioConfig.samplesPerFrame, 0);
     if (decodeLen > 0) {
-        // Use a full memory barrier to ensure the circular buffer is written before incrementing the index
-        __sync_synchronize();
+        // Provide backpressure on the queue to ensure too many frames don't build up
+        // in SDL's audio queue.
+        while (SDL_GetQueuedAudioSize(audioDevice) / audioFrameSize > 10) {
+            SDL_Delay(1);
+        }
         
-        // This can race with the reader in the sample callback, however this is a benign
-        // race since we'll either read the original value of s_WriteIndex (which is safe,
-        // we just won't consider this sample) or the new value of s_WriteIndex
-        audioBufferWriteIndex = (audioBufferWriteIndex + 1) % CIRCULAR_BUFFER_SIZE;
+        if (SDL_QueueAudio(audioDevice,
+                           audioBuffer,
+                           sizeof(short) * decodeLen * audioConfig.channelCount) < 0) {
+            Log(LOG_E, @"Failed to queue audio sample: %s\n", SDL_GetError());
+        }
     }
 }
 
@@ -210,9 +259,9 @@ void ClStageComplete(int stage)
     [_callbacks stageComplete:LiGetStageName(stage)];
 }
 
-void ClStageFailed(int stage, long errorCode)
+void ClStageFailed(int stage, int errorCode)
 {
-    [_callbacks stageFailed:LiGetStageName(stage) withError:errorCode];
+    [_callbacks stageFailed:LiGetStageName(stage) withError:errorCode portTestFlags:LiGetPortFlagsFromStage(stage)];
 }
 
 void ClConnectionStarted(void)
@@ -220,7 +269,7 @@ void ClConnectionStarted(void)
     [_callbacks connectionStarted];
 }
 
-void ClConnectionTerminated(long errorCode)
+void ClConnectionTerminated(int errorCode)
 {
     [_callbacks connectionTerminated: errorCode];
 }
@@ -238,6 +287,11 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
     [_callbacks rumble:controllerNumber lowFreqMotor:lowFreqMotor highFreqMotor:highFreqMotor];
 }
 
+void ClConnectionStatusUpdate(int status)
+{
+    [_callbacks connectionStatusUpdate:status];
+}
+
 -(void) terminate
 {
     // Interrupt any action blocking LiStartConnection(). This is
@@ -266,6 +320,10 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
         initLock = [[NSLock alloc] init];
     }
     
+    if (videoStatsLock == nil) {
+        videoStatsLock = [[NSLock alloc] init];
+    }
+    
     strncpy(_hostString,
             [config.host cStringUsingEncoding:NSUTF8StringEncoding],
             sizeof(_hostString));
@@ -277,6 +335,11 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
                 [config.gfeVersion cStringUsingEncoding:NSUTF8StringEncoding],
                 sizeof(_gfeVersionString));
     }
+    if (config.rtspSessionUrl != nil) {
+        strncpy(_rtspSessionUrl,
+                [config.rtspSessionUrl cStringUsingEncoding:NSUTF8StringEncoding],
+                sizeof(_rtspSessionUrl));
+    }
 
     LiInitializeServerInformation(&_serverInfo);
     _serverInfo.address = _hostString;
@@ -284,6 +347,9 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
     if (config.gfeVersion != nil) {
         _serverInfo.serverInfoGfeVersion = _gfeVersionString;
     }
+    if (config.rtspSessionUrl != nil) {
+        _serverInfo.rtspSessionUrl = _rtspSessionUrl;
+    }
 
     renderer = myRenderer;
     _callbacks = callbacks;
@@ -294,49 +360,50 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
     _streamConfig.fps = config.frameRate;
     _streamConfig.bitrate = config.bitRate;
     _streamConfig.enableHdr = config.enableHdr;
+    _streamConfig.audioConfiguration = config.audioConfiguration;
+    
+    // TODO: If/when video encryption is added, we'll probably want to
+    // limit that to devices that support the ARMv8 AES instructions.
+    _streamConfig.encryptionFlags = ENCFLG_AUDIO;
     
     // Use some of the HEVC encoding efficiency improvements to
     // reduce bandwidth usage while still gaining some image
     // quality improvement.
     _streamConfig.hevcBitratePercentageMultiplier = 75;
     
-    // Detect remote streaming automatically based on the IP address of the target
-    _streamConfig.streamingRemotely = STREAM_CFG_AUTO;
-    _streamConfig.packetSize = 1392;
-    
-    switch (config.audioChannelCount) {
-        case 2:
-            _streamConfig.audioConfiguration = AUDIO_CONFIGURATION_STEREO;
-            break;
-        case 6:
-            _streamConfig.audioConfiguration = AUDIO_CONFIGURATION_51_SURROUND;
-            break;
-        default:
-            Log(LOG_E, @"Unknown audio channel count: %d", config.audioChannelCount);
-            abort();
+    if ([Utils isActiveNetworkVPN]) {
+        // Force remote streaming mode when a VPN is connected
+        _streamConfig.streamingRemotely = STREAM_CFG_REMOTE;
+        _streamConfig.packetSize = 1024;
+    }
+    else {
+        // Detect remote streaming automatically based on the IP address of the target
+        _streamConfig.streamingRemotely = STREAM_CFG_AUTO;
+        _streamConfig.packetSize = 1392;
     }
     
     // HDR implies HEVC allowed
     if (config.enableHdr) {
         config.allowHevc = YES;
     }
+    
+    // Streaming at resolutions above 4K requires HEVC
+    if (config.width > 4096 || config.height > 4096) {
+        config.allowHevc = YES;
+    }
 
-#if TARGET_OS_IPHONE
     // On iOS 11, we can use HEVC if the server supports encoding it
     // and this device has hardware decode for it (A9 and later).
     // Additionally, iPhone X had a bug which would cause video
     // to freeze after a few minutes with HEVC prior to iOS 11.3.
     // As a result, we will only use HEVC on iOS 11.3 or later.
-    if (@available(iOS 11.3, *)) {
-        _streamConfig.supportsHevc = config.allowHevc && VTIsHardwareDecodeSupported(kCMVideoCodecType_HEVC);
-    }
-#else
-    if (@available(macOS 10.13, *)) {
-        // Streaming with limited bandwidth will result in better quality with HEVC
-        if (VTIsHardwareDecodeSupported(kCMVideoCodecType_HEVC) || _streamConfig.streamingRemotely != 0)
-            _streamConfig.supportsHevc = config.allowHevc;
-    }
+    if (@available(iOS 11.3, tvOS 11.3, *)) {
+        _streamConfig.supportsHevc =
+#if !TARGET_OS_TV
+            config.allowHevc &&
 #endif
+            VTIsHardwareDecodeSupported(kCMVideoCodecType_HEVC);
+    }
     
     // HEVC must be supported when HDR is enabled
     assert(!_streamConfig.enableHdr || _streamConfig.supportsHevc);
@@ -348,18 +415,23 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
 
     LiInitializeVideoCallbacks(&_drCallbacks);
     _drCallbacks.setup = DrDecoderSetup;
-    _drCallbacks.submitDecodeUnit = DrSubmitDecodeUnit;
+    _drCallbacks.cleanup = DrCleanup;
 
     // RFI doesn't work properly with HEVC on iOS 11 with an iPhone SE (at least)
     // It doesnt work on macOS either, tested with Network Link Conditioner.
-    _drCallbacks.capabilities = CAPABILITY_REFERENCE_FRAME_INVALIDATION_AVC |
-                                CAPABILITY_DIRECT_SUBMIT;
+    // RFI seems to be broken at all resolutions on the Apple TV 4K (1st gen)
+    // on tvOS 14.5.
+    _drCallbacks.capabilities =
+#if !TARGET_OS_TV
+                                CAPABILITY_REFERENCE_FRAME_INVALIDATION_AVC |
+#endif
+                                CAPABILITY_PULL_RENDERER;
 
     LiInitializeAudioCallbacks(&_arCallbacks);
     _arCallbacks.init = ArInit;
     _arCallbacks.cleanup = ArCleanup;
     _arCallbacks.decodeAndPlaySample = ArDecodeAndPlaySample;
-    _arCallbacks.capabilities = CAPABILITY_DIRECT_SUBMIT;
+    _arCallbacks.capabilities = CAPABILITY_SUPPORTS_ARBITRARY_AUDIO_DURATION;
 
     LiInitializeConnectionCallbacks(&_clCallbacks);
     _clCallbacks.stageStarting = ClStageStarting;
@@ -369,39 +441,11 @@ void ClRumble(unsigned short controllerNumber, unsigned short lowFreqMotor, unsi
     _clCallbacks.connectionTerminated = ClConnectionTerminated;
     _clCallbacks.logMessage = ClLogMessage;
     _clCallbacks.rumble = ClRumble;
+    _clCallbacks.connectionStatusUpdate = ClConnectionStatusUpdate;
 
     return self;
 }
 
-static void FillOutputBuffer(void *aqData,
-                             AudioQueueRef inAQ,
-                             AudioQueueBufferRef inBuffer) {
-    inBuffer->mAudioDataByteSize = activeChannelCount * FRAME_SIZE * sizeof(short);
-    
-    assert(inBuffer->mAudioDataByteSize == inBuffer->mAudioDataBytesCapacity);
-    
-    // If the indexes aren't equal, we have a sample
-    if (audioBufferWriteIndex != audioBufferReadIndex) {
-        // Copy data to the audio buffer
-        memcpy(inBuffer->mAudioData,
-               audioCircularBuffer[audioBufferReadIndex],
-               inBuffer->mAudioDataByteSize);
-        
-        // Use a full memory barrier to ensure the circular buffer is read before incrementing the index
-        __sync_synchronize();
-        
-        // This can race with the reader in the AudDecDecodeAndPlaySample function. This is
-        // not a problem because at worst, it just won't see that we've consumed this sample yet.
-        audioBufferReadIndex = (audioBufferReadIndex + 1) % CIRCULAR_BUFFER_SIZE;
-    }
-    else {
-        // No data, so play silence
-        memset(inBuffer->mAudioData, 0, inBuffer->mAudioDataByteSize);
-    }
-    
-    AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
-}
-
 -(void) main
 {
     [initLock lock];

Limelight/Stream/ConnectionCallbacks.h

@@ -0,0 +1,21 @@
+//
+//  ConnectionCallbacks.h
+//  Moonlight
+//
+//  Created by Cameron Gutman on 11/1/20.
+//  Copyright © 2020 Moonlight Game Streaming Project. All rights reserved.
+//
+
+@protocol ConnectionCallbacks <NSObject>
+
+- (void) connectionStarted;
+- (void) connectionTerminated:(int)errorCode;
+- (void) stageStarting:(const char*)stageName;
+- (void) stageComplete:(const char*)stageName;
+- (void) stageFailed:(const char*)stageName withError:(int)errorCode portTestFlags:(int)portTestFlags;
+- (void) launchFailed:(NSString*)message;
+- (void) rumble:(unsigned short)controllerNumber lowFreqMotor:(unsigned short)lowFreqMotor highFreqMotor:(unsigned short)highFreqMotor;
+- (void) connectionStatusUpdate:(int)status;
+- (void) videoContentShown;
+
+@end

Limelight/Stream/StreamConfiguration.h

@@ -13,18 +13,17 @@
 @property NSString* gfeVersion;
 @property NSString* appID;
 @property NSString* appName;
+@property NSString* rtspSessionUrl;
 @property int width;
 @property int height;
 @property int frameRate;
 @property int bitRate;
 @property int riKeyId;
-@property BOOL streamingRemotely;
 @property NSData* riKey;
 @property int gamepadMask;
 @property BOOL optimizeGameSettings;
 @property BOOL playAudioOnPC;
-@property int audioChannelCount;
-@property int audioChannelMask;
+@property int audioConfiguration;
 @property BOOL enableHdr;
 @property BOOL multiController;
 @property BOOL allowHevc;

Limelight/Stream/StreamConfiguration.m

@@ -9,5 +9,5 @@
 #import "StreamConfiguration.h"
 
 @implementation StreamConfiguration
-@synthesize host, appID, width, height, frameRate, bitRate, riKeyId, riKey, gamepadMask, streamingRemotely, appName, optimizeGameSettings, playAudioOnPC, audioChannelMask, audioChannelCount, enableHdr, multiController, allowHevc, serverCert;
+@synthesize host, appID, width, height, frameRate, bitRate, riKeyId, riKey, gamepadMask, appName, optimizeGameSettings, playAudioOnPC, audioConfiguration, enableHdr, multiController, allowHevc, serverCert, rtspSessionUrl;
 @end

Limelight/Stream/StreamManager.h

@@ -11,8 +11,10 @@
 
 @interface StreamManager : NSOperation
 
-- (id) initWithConfig:(StreamConfiguration*)config renderView:(OSView*)view connectionCallbacks:(id<ConnectionCallbacks>)callback;
+- (id) initWithConfig:(StreamConfiguration*)config renderView:(UIView*)view connectionCallbacks:(id<ConnectionCallbacks>)callback;
 
 - (void) stopStream;
 
+- (NSString*) getStatsOverlayText;
+
 @end

Limelight/Stream/StreamManager.m

@@ -17,15 +17,17 @@
 #import "HttpRequest.h"
 #import "IdManager.h"
 
+#include <Limelight.h>
+
 @implementation StreamManager {
     StreamConfiguration* _config;
 
-    OSView* _renderView;
+    UIView* _renderView;
     id<ConnectionCallbacks> _callbacks;
     Connection* _connection;
 }
 
-- (id) initWithConfig:(StreamConfiguration*)config renderView:(OSView*)view connectionCallbacks:(id<ConnectionCallbacks>)callbacks {
+- (id) initWithConfig:(StreamConfiguration*)config renderView:(UIView*)view connectionCallbacks:(id<ConnectionCallbacks>)callbacks {
     self = [super init];
     _config = config;
     _renderView = view;
@@ -50,7 +52,11 @@
     NSString* appversion = [serverInfoResp getStringTag:@"appversion"];
     NSString* gfeVersion = [serverInfoResp getStringTag:@"GfeVersion"];
     NSString* serverState = [serverInfoResp getStringTag:@"state"];
-    if (![serverInfoResp isStatusOk] || pairStatus == NULL || appversion == NULL || serverState == NULL) {
+    if (![serverInfoResp isStatusOk]) {
+        [_callbacks launchFailed:serverInfoResp.statusMessage];
+        return;
+    }
+    else if (pairStatus == NULL || appversion == NULL || serverState == NULL) {
         [_callbacks launchFailed:@"Failed to connect to PC"];
         return;
     }
@@ -61,27 +67,32 @@
         return;
     }
     
+    if (_config.width > 4096 || _config.height > 4096) {
+        // Pascal added support for 8K HEVC encoding support. Maxwell 2 could encode HEVC but only up to 4K.
+        // We can't directly identify Pascal, but we can look for HEVC Main10 which was added in the same generation.
+        NSString* codecSupport = [serverInfoResp getStringTag:@"ServerCodecModeSupport"];
+        if (codecSupport == nil || !([codecSupport intValue] & 0x200)) {
+            [_callbacks launchFailed:@"Your host PC's GPU doesn't support streaming video resolutions over 4K."];
+            return;
+        }
+    }
+    
     // resumeApp and launchApp handle calling launchFailed
+    NSString* sessionUrl;
     if ([serverState hasSuffix:@"_SERVER_BUSY"]) {
         // App already running, resume it
-        if (![self resumeApp:hMan]) {
+        if (![self resumeApp:hMan receiveSessionUrl:&sessionUrl]) {
             return;
         }
     } else {
         // Start app
-        if (![self launchApp:hMan]) {
+        if (![self launchApp:hMan receiveSessionUrl:&sessionUrl]) {
             return;
         }
     }
-
-#if TARGET_OS_IPHONE
-    // Set mouse delta factors from the screen resolution and stream size
-    CGFloat screenScale = [[UIScreen mainScreen] scale];
-    CGRect screenBounds = [[UIScreen mainScreen] bounds];
-    CGSize screenSize = CGSizeMake(screenBounds.size.width * screenScale, screenBounds.size.height * screenScale);
-    [((StreamView*)_renderView) setMouseDeltaFactors:_config.width / screenSize.width
-                                                   y:_config.height / screenSize.height];
-#endif
+    
+    // Populate RTSP session URL from launch/resume response
+    _config.rtspSessionUrl = sessionUrl;
     
     // Populate the config's version fields from serverinfo
     _config.appVersion = appversion;
@@ -89,7 +100,7 @@
     
     // Initializing the renderer must be done on the main thread
     dispatch_async(dispatch_get_main_queue(), ^{
-        VideoDecoderRenderer* renderer = [[VideoDecoderRenderer alloc] initWithView:self->_renderView];
+        VideoDecoderRenderer* renderer = [[VideoDecoderRenderer alloc] initWithView:self->_renderView callbacks:self->_callbacks];
         self->_connection = [[Connection alloc] initWithConfig:self->_config renderer:renderer connectionCallbacks:self->_callbacks];
         NSOperationQueue* opQueue = [[NSOperationQueue alloc] init];
         [opQueue addOperation:self->_connection];
@@ -101,7 +112,7 @@
     [_connection terminate];
 }
 
-- (BOOL) launchApp:(HttpManager*)hMan {
+- (BOOL) launchApp:(HttpManager*)hMan receiveSessionUrl:(NSString**)sessionUrl {
     HttpResponse* launchResp = [[HttpResponse alloc] init];
     [hMan executeRequestSynchronously:[HttpRequest requestForResponse:launchResp withUrlRequest:[hMan newLaunchRequest:_config]]];
     NSString *gameSession = [launchResp getStringTag:@"gamesession"];
@@ -115,10 +126,11 @@
         return FALSE;
     }
     
+    *sessionUrl = [launchResp getStringTag:@"sessionUrl0"];
     return TRUE;
 }
 
-- (BOOL) resumeApp:(HttpManager*)hMan {
+- (BOOL) resumeApp:(HttpManager*)hMan receiveSessionUrl:(NSString**)sessionUrl {
     HttpResponse* resumeResp = [[HttpResponse alloc] init];
     [hMan executeRequestSynchronously:[HttpRequest requestForResponse:resumeResp withUrlRequest:[hMan newResumeRequest:_config]]];
     NSString* resume = [resumeResp getStringTag:@"resume"];
@@ -132,7 +144,38 @@
         return FALSE;
     }
     
+    *sessionUrl = [resumeResp getStringTag:@"sessionUrl0"];
     return TRUE;
 }
 
+- (NSString*) getStatsOverlayText {
+    video_stats_t stats;
+    
+    if (!_connection) {
+        return nil;
+    }
+    
+    if (![_connection getVideoStats:&stats]) {
+        return nil;
+    }
+    
+    uint32_t rtt, variance;
+    NSString* latencyString;
+    if (LiGetEstimatedRttInfo(&rtt, &variance)) {
+        latencyString = [NSString stringWithFormat:@"%u ms (variance: %u ms)", rtt, variance];
+    }
+    else {
+        latencyString = @"N/A";
+    }
+    
+    float interval = stats.endTime - stats.startTime;
+    return [NSString stringWithFormat:@"Video stream: %dx%d %.2f FPS (Codec: %@)\nFrames dropped by your network connection: %.2f%%\nAverage network latency: %@",
+            _config.width,
+            _config.height,
+            stats.totalFrames / interval,
+            [_connection getActiveCodecName],
+            stats.networkDroppedFrames / interval,
+            latencyString];
+}
+
 @end

Limelight/Stream/VideoDecoderRenderer.h

@@ -8,14 +8,18 @@
 
 @import AVFoundation;
 
+#import "ConnectionCallbacks.h"
+
 @interface VideoDecoderRenderer : NSObject
 
-- (id)initWithView:(OSView*)view;
+- (id)initWithView:(UIView*)view callbacks:(id<ConnectionCallbacks>)callbacks;
 
-- (void)setupWithVideoFormat:(int)videoFormat;
+- (void)setupWithVideoFormat:(int)videoFormat refreshRate:(int)refreshRate;
+
+- (void)cleanup;
 
 - (void)updateBufferForRange:(CMBlockBufferRef)existingBuffer data:(unsigned char *)data offset:(int)offset length:(int)nalLength;
 
-- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType;
+- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType frameType:(int)frameType pts:(unsigned int)pts;
 
 @end

Limelight/Stream/VideoDecoderRenderer.m

@@ -13,6 +13,7 @@
 
 @implementation VideoDecoderRenderer {
     StreamView* _view;
+    id<ConnectionCallbacks> _callbacks;
     
     AVSampleBufferDisplayLayer* displayLayer;
     Boolean waitingForSps, waitingForPps, waitingForVps;
@@ -20,6 +21,8 @@
     
     NSData *spsData, *ppsData, *vpsData;
     CMVideoFormatDescriptionRef formatDesc;
+    
+    CADisplayLink* _displayLink;
 }
 
 - (void)reinitializeDisplayLayer
@@ -28,7 +31,7 @@
     
     displayLayer = [[AVSampleBufferDisplayLayer alloc] init];
     displayLayer.bounds = _view.bounds;
-    displayLayer.backgroundColor = [OSColor blackColor].CGColor;
+    displayLayer.backgroundColor = [UIColor blackColor].CGColor;
     
     displayLayer.position = CGPointMake(CGRectGetMidX(_view.bounds), CGRectGetMidY(_view.bounds));
     displayLayer.videoGravity = AVLayerVideoGravityResizeAspect;
@@ -59,23 +62,52 @@
     }
 }
 
-- (id)initWithView:(StreamView*)view
+- (id)initWithView:(StreamView*)view callbacks:(id<ConnectionCallbacks>)callbacks
 {
     self = [super init];
     
     _view = view;
+    _callbacks = callbacks;
     
     [self reinitializeDisplayLayer];
     
     return self;
 }
 
-- (void)setupWithVideoFormat:(int)videoFormat
+- (void)setupWithVideoFormat:(int)videoFormat refreshRate:(int)refreshRate
 {
     self->videoFormat = videoFormat;
-#if !TARGET_OS_IPHONE
-    _view.codec = videoFormat;
+    
+    _displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(displayLinkCallback:)];
+    if (@available(iOS 10.0, tvOS 10.0, *)) {
+        _displayLink.preferredFramesPerSecond = refreshRate;
+    }
+    [_displayLink addToRunLoop:[NSRunLoop mainRunLoop] forMode:NSDefaultRunLoopMode];
+}
+                     
+int DrSubmitDecodeUnit(PDECODE_UNIT decodeUnit);
+
+- (void)displayLinkCallback:(CADisplayLink *)sender
+{
+    VIDEO_FRAME_HANDLE handle;
+    PDECODE_UNIT du;
+    
+    while (LiPollNextVideoFrame(&handle, &du)) {
+        LiCompleteVideoFrame(handle, DrSubmitDecodeUnit(du));
+        
+#if 0
+        // Always keep one pending frame smooth out gaps due to
+        // network jitter at the cost of 1 frame of latency
+        if (LiGetPendingVideoFrames() == 1) {
+            break;
+        }
 #endif
+    }
+}
+
+- (void)cleanup
+{
+    [_displayLink invalidate];
 }
 
 #define FRAME_START_PREFIX_SIZE 4
@@ -93,27 +125,6 @@
     }
 }
 
-- (Boolean)isNalReferencePicture:(unsigned char)nalType
-{
-    if (videoFormat & VIDEO_FORMAT_MASK_H264) {
-        return nalType == 0x65;
-    }
-    else {
-        // HEVC has several types of reference NALU types
-        switch (nalType) {
-            case 0x20:
-            case 0x22:
-            case 0x24:
-            case 0x26:
-            case 0x28:
-            case 0x2A:
-                return true;
-            default:
-                return false;
-        }
-    }
-}
-
 - (void)updateBufferForRange:(CMBlockBufferRef)existingBuffer data:(unsigned char *)data offset:(int)offset length:(int)nalLength
 {
     OSStatus status;
@@ -181,9 +192,8 @@
 }
 
 // This function must free data for bufferType == BUFFER_TYPE_PICDATA
-- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType
+- (int)submitDecodeBuffer:(unsigned char *)data length:(int)length bufferType:(int)bufferType frameType:(int)frameType pts:(unsigned int)pts
 {
-    unsigned char nalType = data[FRAME_START_PREFIX_SIZE];
     OSStatus status;
     
     if (bufferType != BUFFER_TYPE_PICDATA) {
@@ -232,11 +242,7 @@
                 
                 Log(LOG_I, @"Constructing new HEVC format description");
 
-#if TARGET_OS_IPHONE
                 if (@available(iOS 11.0, *)) {
-#else
-                if (@available(macOS 10.13, *)) {
-#endif
                     status = CMVideoFormatDescriptionCreateFromHEVCParameterSets(kCFAllocatorDefault,
                                                                                  3, /* count of parameter sets */
                                                                                  parameterSetPointers,
@@ -337,7 +343,7 @@
     CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
     CFDictionarySetValue(dict, kCMSampleAttachmentKey_IsDependedOnByOthers, kCFBooleanTrue);
     
-    if (![self isNalReferencePicture:nalType]) {
+    if (frameType == FRAME_TYPE_PFRAME) {
         // P-frame
         CFDictionarySetValue(dict, kCMSampleAttachmentKey_NotSync, kCFBooleanTrue);
         CFDictionarySetValue(dict, kCMSampleAttachmentKey_DependsOnOthers, kCFBooleanTrue);
@@ -347,22 +353,20 @@
         CFDictionarySetValue(dict, kCMSampleAttachmentKey_DependsOnOthers, kCFBooleanFalse);
     }
 
-#if !TARGET_OS_IPHONE
-    _view.frameCount++;
-#endif
-
-    // Enqueue video samples on the main thread
-    dispatch_async(dispatch_get_main_queue(), ^{
-        // Enqueue the next frame
-        [self->displayLayer enqueueSampleBuffer:sampleBuffer];
-        
+    // Enqueue the next frame
+    [self->displayLayer enqueueSampleBuffer:sampleBuffer];
+    
+    if (frameType == FRAME_TYPE_IDR) {
         // Ensure the layer is visible now
         self->displayLayer.hidden = NO;
         
-        // Dereference the buffers
-        CFRelease(blockBuffer);
-        CFRelease(sampleBuffer);
-    });
+        // Tell our parent VC to hide the progress indicator
+        [self->_callbacks videoContentShown];
+    }
+    
+    // Dereference the buffers
+    CFRelease(blockBuffer);
+    CFRelease(sampleBuffer);
     
     return DR_OK;
 }

Limelight/UIAppView.h

@@ -11,11 +11,16 @@
 
 @protocol AppCallback <NSObject>
 
-- (void) appClicked:(TemporaryApp*) app;
+- (void) appClicked:(TemporaryApp*)app view:(UIView*)view;
+- (void) appLongClicked:(TemporaryApp*)app view:(UIView*)view;
 
 @end
 
-@interface UIAppView : UIView
+#if !TARGET_OS_TV
+@interface UIAppView : UIButton <UIContextMenuInteractionDelegate>
+#else
+@interface UIAppView : UIButton
+#endif
 
 - (id) initWithApp:(TemporaryApp*)app cache:(NSCache*)cache andCallback:(id<AppCallback>)callback;
 - (void) updateAppImage;

Limelight/UIAppView.m

@@ -9,13 +9,13 @@
 #import "UIAppView.h"
 #import "AppAssetManager.h"
 
-static const float REFRESH_CYCLE = 2.0f;
+static const float REFRESH_CYCLE = 1.0f;
 
 @implementation UIAppView {
     TemporaryApp* _app;
-    UIButton* _appButton;
     UILabel* _appLabel;
     UIImageView* _appOverlay;
+    UIImageView* _appImage;
     NSCache* _artCache;
     id<AppCallback> _callback;
 }
@@ -33,67 +33,95 @@ static UIImage* noImage;
     if (noImage == nil) {
         noImage = [UIImage imageNamed:@"NoAppImage"];
     }
+        
+#if TARGET_OS_TV
+    self.frame = CGRectMake(0, 0, 200, 265);
+#else
+    self.frame = CGRectMake(0, 0, 150, 200);
+#endif
+    
+    [self setAlpha:app.hidden ? 0.4 : 1.0];
+    
+    _appImage = [[UIImageView alloc] initWithFrame:self.frame];
+    [_appImage setImage:noImage];
+    [self addSubview:_appImage];
+    
+    // Use UIContextMenuInteraction on iOS 13.0+ and a standard UILongPressGestureRecognizer
+    // for tvOS devices and iOS prior to 13.0.
+#if !TARGET_OS_TV
+    if (@available(iOS 13.0, *)) {
+        UIContextMenuInteraction* rightClickInteraction = [[UIContextMenuInteraction alloc] initWithDelegate:self];
+        [self addInteraction:rightClickInteraction];
+    }
+    else
+#endif
+    {
+        UILongPressGestureRecognizer* longPressRecognizer = [[UILongPressGestureRecognizer alloc] initWithTarget:self action:@selector(appLongClicked:)];
+        [self addGestureRecognizer:longPressRecognizer];
+    }
+    
+    [self addTarget:self action:@selector(appClicked:) forControlEvents:UIControlEventPrimaryActionTriggered];
+    
+    [self addTarget:self action:@selector(buttonSelected:) forControlEvents:UIControlEventTouchDown];
+    [self addTarget:self action:@selector(buttonDeselected:) forControlEvents:UIControlEventTouchUpInside | UIControlEventTouchCancel | UIControlEventTouchDragExit];
     
 #if TARGET_OS_TV
-    _appButton = [UIButton buttonWithType:UIButtonTypeSystem];
+    _appImage.adjustsImageWhenAncestorFocused = YES;
 #else
-    _appButton = [UIButton buttonWithType:UIButtonTypeCustom];
-#endif
-    [_appButton setBackgroundImage:noImage forState:UIControlStateNormal];
-    [_appButton setContentEdgeInsets:UIEdgeInsetsMake(0, 4, 0, 4)];
-    [_appButton sizeToFit];
-    if (@available(iOS 9.0, tvOS 9.0, *)) {
-        [_appButton addTarget:self action:@selector(appClicked) forControlEvents:UIControlEventPrimaryActionTriggered];
-    }
-    else {
-        [_appButton addTarget:self action:@selector(appClicked) forControlEvents:UIControlEventTouchUpInside];
-    }
-    
-    [self addSubview:_appButton];
-    [self sizeToFit];
-    
     // Rasterizing the cell layer increases rendering performance by quite a bit
+    // but we want it unrasterized for tvOS where it must be scaled.
     self.layer.shouldRasterize = YES;
     self.layer.rasterizationScale = [UIScreen mainScreen].scale;
     
+    if (@available(iOS 13.4.1, *)) {
+        // Allow the button style to change when moused over
+        self.pointerInteractionEnabled = YES;
+    }
+#endif
+    
     [self updateAppImage];
-    [self startUpdateLoop];
     
     return self;
 }
 
-- (void) appClicked {
-    [_callback appClicked:_app];
+- (void)didMoveToSuperview {
+    // Start our update loop when we are added to our cell
+    if (self.superview != nil) {
+        [self updateLoop];
+    }
 }
 
+- (void) appClicked:(UIView *)view {
+    [_callback appClicked:_app view:view];
+}
+
+- (void) appLongClicked:(UILongPressGestureRecognizer*)gesture {
+    if (gesture.state == UIGestureRecognizerStateBegan) {
+        [_callback appLongClicked:_app view:self];
+    }
+}
+
+#if !TARGET_OS_TV
+- (UIContextMenuConfiguration *)contextMenuInteraction:(UIContextMenuInteraction *)interaction
+                        configurationForMenuAtLocation:(CGPoint)location {
+    // We don't want to trigger the primary action at this point, so cancel
+    // tracking touch on this view now. This will also have the (intended)
+    // effect of removing the touch highlight on this view.
+    [self cancelTrackingWithEvent:nil];
+    
+    [_callback appLongClicked:_app view:self];
+    return nil;
+}
+#endif
+
 - (void) updateAppImage {
     if (_appOverlay != nil) {
         [_appOverlay removeFromSuperview];
         _appOverlay = nil;
     }
-    
-    _appButton.frame = CGRectMake(0, 0, noImage.size.width, noImage.size.height);
-    self.frame = CGRectMake(0, 0, noImage.size.width, noImage.size.height);
-    
-    if ([_app.id isEqualToString:_app.host.currentGame]) {
-#if TARGET_OS_TV
-        _appButton.layer.masksToBounds = NO;
-
-        _appButton.layer.shadowColor = [UIColor greenColor].CGColor;
-        _appButton.layer.shadowOffset = CGSizeMake(5,8);
-        _appButton.layer.shadowOpacity = 0.7;
-#else
-        // Only create the app overlay if needed
-        _appOverlay = [[UIImageView alloc] initWithImage:[UIImage imageNamed:@"Play"]];
-        _appOverlay.layer.shadowColor = [UIColor blackColor].CGColor;
-        _appOverlay.layer.shadowOffset = CGSizeMake(0, 0);
-        _appOverlay.layer.shadowOpacity = 1;
-        _appOverlay.layer.shadowRadius = 2.0;
-        [self addSubview:_appOverlay];
-        
-        _appOverlay.frame = CGRectMake(0, 0, noImage.size.width / 2.f, noImage.size.height / 4.f);
-        [_appOverlay setCenter:CGPointMake(self.frame.size.width/2, self.frame.size.height/6)];
-#endif
+    if (_appLabel != nil) {
+        [_appLabel removeFromSuperview];
+        _appLabel = nil;
     }
     
     BOOL noAppImage = false;
@@ -107,31 +135,13 @@ static UIImage* noImage;
             [_artCache setObject:appImage forKey:_app];
         }
     }
+    
     if (appImage != nil) {
         // This size of image might be blank image received from GameStream.
         // TODO: Improve no-app image detection
         if (!(appImage.size.width == 130.f && appImage.size.height == 180.f) && // GFE 2.0
             !(appImage.size.width == 628.f && appImage.size.height == 888.f)) { // GFE 3.0
-#if TARGET_OS_TV
-            //custom image to do TvOS hover popup effect
-            UIImageView *imageView = [[UIImageView alloc] initWithImage:appImage];
-            imageView.userInteractionEnabled = YES;
-            imageView.adjustsImageWhenAncestorFocused = YES;
-            imageView.frame = CGRectMake(0, 0, 200, 265);
-            [_appButton addSubview:imageView];
-            
-            _appButton.frame = CGRectMake(0, 0, 200, 265);
-            self.frame = CGRectMake(0, 0, 200, 265);
-#else
-            _appButton.frame = CGRectMake(0, 0, appImage.size.width / 2, appImage.size.height / 2);
-            self.frame = CGRectMake(0, 0, appImage.size.width / 2, appImage.size.height / 2);
-
-            _appOverlay.frame = CGRectMake(0, 0, self.frame.size.width / 2.f, self.frame.size.height / 4.f);
-            _appOverlay.layer.shadowRadius = 4.0;
-            [_appOverlay setCenter:CGPointMake(self.frame.size.width/2, self.frame.size.height/6)];
-#endif
-            [_appButton setBackgroundImage:appImage forState:UIControlStateNormal];
-            [self setNeedsDisplay];
+            [_appImage setImage:appImage];
         } else {
             noAppImage = true;
         }
@@ -139,58 +149,89 @@ static UIImage* noImage;
         noAppImage = true;
     }
     
+    if ([_app.id isEqualToString:_app.host.currentGame]) {
+        // Only create the app overlay if needed
+        _appOverlay = [[UIImageView alloc] initWithImage:[UIImage imageNamed:@"Play"]];
+        _appOverlay.layer.shadowColor = [UIColor blackColor].CGColor;
+        _appOverlay.layer.shadowOffset = CGSizeMake(0, 0);
+        _appOverlay.layer.shadowOpacity = 1;
+        _appOverlay.layer.shadowRadius = 4.0;
+        _appOverlay.contentMode = UIViewContentModeScaleAspectFit;
+    }
+    
     if (noAppImage) {
         _appLabel = [[UILabel alloc] init];
         [_appLabel setTextColor:[UIColor whiteColor]];
+        [_appLabel setText:_app.name];
+        [_appLabel setFont:[UIFont systemFontOfSize:24]];
         [_appLabel setBaselineAdjustment:UIBaselineAdjustmentAlignCenters];
         [_appLabel setTextAlignment:NSTextAlignmentCenter];
         [_appLabel setLineBreakMode:NSLineBreakByWordWrapping];
         [_appLabel setNumberOfLines:0];
-        [_appLabel setText:_app.name];
-#if TARGET_OS_TV
-        [_appLabel setFont:[UIFont systemFontOfSize:16]];
-        [_appLabel setAdjustsFontSizeToFitWidth:YES];
-        [_appLabel setFrame: CGRectMake(0, 0, 200, 265)];
-        //custom image to do TvOS hover popup effect
-        UIImageView *imageView = [[UIImageView alloc] initWithImage:appImage];
-        imageView.userInteractionEnabled = YES;
-        imageView.adjustsImageWhenAncestorFocused = YES;
-        imageView.frame = CGRectMake(0, 0, 200, 265);
-        UIGraphicsBeginImageContextWithOptions(_appLabel.frame.size, false, 0);
-        [imageView.layer renderInContext:(UIGraphicsGetCurrentContext())];
-        [_appLabel.layer renderInContext:(UIGraphicsGetCurrentContext())];
-        UIImage *imageWithText = UIGraphicsGetImageFromCurrentImageContext();
-        UIGraphicsEndImageContext();
-        [imageView setImage:imageWithText];
-        [_appButton addSubview:imageView];
-        _appButton.frame = CGRectMake(0, 0, 200, 265);
-        self.frame = CGRectMake(0, 0, 200, 265);
-#else
-        CGFloat padding = 4.f;
-        [_appLabel setFrame: CGRectMake(padding, padding, _appButton.frame.size.width - 2 * padding, _appButton.frame.size.height - 2 * padding)];
-        [_appButton addSubview:_appLabel];
-#endif
     }
     
+    [self positionSubviews];
+    
+#if TARGET_OS_TV
+    [_appImage.overlayContentView addSubview:_appLabel];
+    [_appImage.overlayContentView addSubview:_appOverlay];
+#else
+    [self addSubview:_appLabel];
+    [self addSubview:_appOverlay];
+#endif
 }
 
-- (void) startUpdateLoop {
-    [self performSelector:@selector(updateLoop) withObject:self afterDelay:REFRESH_CYCLE];
+- (void) buttonSelected:(id)sender {
+    _appImage.layer.opacity = 0.5f;
+}
+- (void) buttonDeselected:(id)sender {
+    _appImage.layer.opacity = 1.0f;
+}
+
+- (void) positionSubviews {
+    CGFloat padding = 5.f;
+    CGSize frameSize = _appImage.frame.size;
+    CGPoint center = _appImage.center;
+    
+    if (_appLabel != nil) {
+        if (_appOverlay != nil) {
+            _appOverlay.frame = CGRectMake(0, 0, frameSize.width / 3, frameSize.width / 3);
+            _appOverlay.center = CGPointMake(frameSize.width / 2, padding + _appOverlay.frame.size.height / 2);
+            
+            [_appLabel setFrame:CGRectMake(padding, _appOverlay.frame.size.height + padding, frameSize.width - 2 * padding, frameSize.height - _appOverlay.frame.size.height - 2 * padding)];
+        }
+        else {
+            [_appLabel setFrame:CGRectMake(padding, padding, frameSize.width - 2 * padding, frameSize.height - 2 * padding)];
+        }
+    }
+    else if (_appOverlay != nil) {
+        _appOverlay.frame = CGRectMake(0, 0, frameSize.width / 2, frameSize.width / 2);
+        _appOverlay.center = center;
+    }
 }
 
 - (void) updateLoop {
+    // Stop immediately if the view has been detached
+    if (self.superview == nil) {
+        return;
+    }
+    
     // Update the app image if neccessary
     if ((_appOverlay != nil && ![_app.id isEqualToString:_app.host.currentGame]) ||
         (_appOverlay == nil && [_app.id isEqualToString:_app.host.currentGame])) {
-#if !TARGET_OS_TV
         [self updateAppImage];
-#endif
     }
     
-    // Stop updating when we detach from our parent view
-    if (self.superview != nil) {
-        [self performSelector:@selector(updateLoop) withObject:self afterDelay:REFRESH_CYCLE];
-    }
+    // Show no shadow for hidden apps. Because we adjust the opacity of the
+    // cells for hidden apps, it makes them look bad when the shadow draws
+    // through the app tile.
+    self.superview.layer.shadowOpacity = _app.hidden ? 0.0f : 0.5f;
+    
+    // Update opacity if neccessary
+    [self setAlpha:_app.hidden ? 0.4 : 1.0];
+    
+    // Queue the next refresh cycle
+    [self performSelector:@selector(updateLoop) withObject:self afterDelay:REFRESH_CYCLE];
 }
 
 @end

Limelight/UIComputerView.h

@@ -17,7 +17,11 @@
 
 @end
 
-@interface UIComputerView : UIView
+#if !TARGET_OS_TV
+@interface UIComputerView : UIButton <UIContextMenuInteractionDelegate>
+#else
+@interface UIComputerView : UIButton
+#endif
 
 - (id) initWithComputer:(TemporaryHost*)host andCallback:(id<HostCallback>)callback;
 - (id) initForAddWithCallback:(id<HostCallback>)callback;