Microsoft builds on Recall with Gaming Copilot — fails basic privacy tests
doublepulsar.com
Gaming Copilot, rolling out now to Windows 11, adds a new attack surface to Windows.

The software is labelled Beta, but is being pushed out to existing PCs anyway because… well… why not beta something in prod, it’s 2025 after all.

It appears in Game Bar, which is accessible from Windows Key + G. You may not have it yet as they are staging deployment. I’m not in Windows Insiders, but I have it installed — so it looks like the roll out is becoming wide. Note that I had uninstalled Copilot from my PC… but Gaming Copilot silently installed anyway.

It is similar to Recall, except not all the processing is done locally — it relies on the cloud. It screenshots gameplay, and then extracts elements of the screen (such as symbols and text) to work out what the player is doing. The idea is it can help you game, e.g. you can ask questions about what you’re doing in the game at a given moment.

  • utopiah@lemmy.worldEnglish
    11·
    5 days ago

    If someone somehow wants to test this locally I suggest

    • install locally a vision model, e.g. Moondream (which Ollama supports but alternatives too), then
    • take a screenshot of your game,
    • write a prompt like “How can I play this game better”
    • query the vision model with the image and your prompt

    marvel at how pointless and costly the whole setup is and how a basic query on e.g. DuckDuckGo with “game name” + prompt would yield way WAY better results from actual human, uninstall the whole, keep on playing with your actual brain.

    At least now you can say you tried before you complain, rightfully, that it sucks.

    For more check https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence

    PS: I didn’t actually try this, I’m too lazy for that right how but feel free to report back if you do!

    Edit : 2 potential optimization (despite not being sure it ever makes sense in the first place!)

    • do so automatically, e.g. ~/gaming_screenshots directory (via e.g. Spectacle shortcut) monitored via inotify then notify-send the suggestion, thus stay in game during the whole process
    • fine tune on specific visual datasets, e.g. rely on fextra as mentioned in https://lemmy.world/post/37758804/20113877
      • utopiah@lemmy.worldEnglish
        3·
        6 days ago

        Spot on, it’s a variation of the “Robots were supposed to clean the house why I make art, not the other way around” meme and arguably Moravec’s paradox.