Fantus-button part 2: the physical button build and the network communication

First part of this series is here, covering the reverse engineering of the DRTV Chromecast App.

I wanted the physical appearance to be extremely minimalistic, with slight references to various cubes from videogames. Because it is a remote control, it of course has to be wireless and battery-powered.

The box is lasercut from 6 mm MDF, and with a giant red arcade button on top with a red LED inside.

The electronics inside is a battery-powered Wemos D1, along with 4 x 18650 Lithium battery cells. After some experimentation on the response time, which is primarily dominated by the time it takes to reconnect to the WiFi network, I initially only used “light sleep”. This resulted in a battery time of just over a week, which is okay, but not great.

In order to preserve battery deep sleep would be really nice. The problem is deep sleep on the Wemos can only be interrupted by a reset. The idea was to use a MOSFET (in this case an N-channel logic level mosfet, IRFZ44N) for the Wemos to be able to select whether a press of the button should reset it, or it should just register on a pin as normal.

This circuit allows RST to be pulled low by the button, as long as D0 is high. Luckily, D0 is high during deep sleep, so as long as the Arduino code keeps D0 low button presses will not reset — but can still be registered by reading pin D1.

This works out “responsively enough” because the initial start has some delay due to the Chromecast initializing the app and loading media. Any subsequent button presses within the 30 seconds the Arduino stays awake are instant though. With this setup the battery life is not a problem – I’ve only had to charge it once. As a bonus feature/bug whenever the battery gets low the Wemos will trigger a bit sporadically: this causes “Fantus-bombing” where Fantus will just randomly start; quite quickly thereafter the Fantus-button is being charged 😉

The Wemos itself is not powerful enough to do all the pyChromecast communication needed, so I setup a small Raspberry Pi to handle that part. Since I didn’t want to spend too much time and effort setting up the communication between them, I ended up using a trick from my youth: UDP broadcasting. Because UDP is datagram-oriented you can send a UDP packet to the broadcast address (255.255.255.255) and then it will be received by all hosts on the local area network: no configuration needed. In Arduino code it looks like:

  Udp.begin(31337);
  Udp.beginPacket("255.255.255.255", 31337);
  Udp.write("emergency-button\n");
  Udp.endPacket();

(Full Arduino code available here.)

At this point I had a UDP packet that I could receive on the Raspberry Pi, and it was just a matter of writing a small server program to listen, receive and process those UDP commands. However, at this point a thought entered my mind, that derailed the project for a while:

netcat | bash

Why write my own server to parse and execute commands, when Bash is already fully capable of doing exactly that with more flexibility than I could ever dream of? And netcat is perfectly capable of receiving UDP packets? This is a UNIX system, after all, and UNIX is all about combining simple commands in pipelines — each doing one thing well.

The diabolical simplicity of just executing commands directly from the network was a bit too insecure though. This is where Bash Restricted mode enters the project: I wouldn’t rely on it for high security (since it is trying to “enumerate badness“), but by locking down the PATH of commands that are allowed to execute it should be relatively safe from most of the common bypass techniques:

netcat -u -k -l 31337 | PATH=./handlers/ /bin/bash -r

The project was now fully working: press the button, Fantus starts. Press it while Fantus is playing: Fantus pauses. Press it while Fantus is paused: Fantus resumes. The little human was delighted about his new powers over the world, and pressed the button to his hearts content (and his parents slight annoyance at times).

(Full code for handler available here.)

But wouldn’t it be cool if the little human had a (limited) choice in what to view?…

Fantus-button part 1: Reverse engineering the DRTV Chromecast App

I want to build a physical giant red button, that when pressed instantly starts a children’s TV-show, in my case Fantus on DRTV using a Chromecast.

The first part of the build is figuring out how to remotely start a specific video on a Chromecast. Initially I thought this would be pretty simple to do from an Arduino, because back in the day you could start a video just using a HTTP request. Very much not so anymore: the Chromecast protocol has evolved into some monster using JSON inside Protobuf over TLS/TCP, with multicast DNS for discovery. Chance of getting that working on a microcontroller is near-zero.

But remote control is possible using e.g. pychromecast which has support for not only the usual app of YouTube, but also a couple of custom ones like BBC. Let’s try and add support for DRTV to pychromecast, starting at the hints given on adding a new app.

Using the netlog-viewer to decode the captured net-export from Chrome, and looking at the unencrypted socket communication, the appId of the DRTV app is easily found.

However, one of the subsequent commands has a lot more customData than I expected, since it should more or less just be the contentId that is needed:

{
  "items": [
    {
      "autoplay": true,
      "customData": {
        "accountToken": {
          "expirationDate": "2022-07-02T00:48:35.391Z",
          "geoLocation": "dk",
          "isCountryVerified": false,
          "isDeviceAbroad": false,
          "isFallbackToken": false,
          "isOptedOut": false,
          "profileId": "c4e0...f3e",
          "refreshable": true,
          "scope": "Catalog",
          "type": "UserAccount",
          "value": "eyJ0eX...Dh8kXg"
        },
        "chainPlayCountdown": 10,
        "profileToken": {
          "expirationDate": "2022-07-02T00:48:35.389Z",
          "geoLocation": "dk",
          "isCountryVerified": false,
          "isDeviceAbroad": false,
          "isFallbackToken": false,
          "isOptedOut": false,
          "profileId": "c4e0a...f3e",
          "refreshable": true,
          "scope": "Catalog",
          "type": "UserProfile",
          "value": "eyJ0eXAi...IkWOU5TA"
        },
        "senderAppVersion": "2.211.33",
        "senderDeviceType": "web_browser",
        "sessionId": "cd84eb44-bce0-495b-ab6a-41ef125b945d",
        "showDebugOverlay": false,
        "userId": ""
      },
      "media": {
        "contentId": "278091",
        "contentType": "video/hls",
        "customData": {
          "accessService": "StandardVideo"
        },
        "streamType": "BUFFERED"
      },
      "preloadTime": 0,
      "startTime": 0
    }
  ],
  "repeatMode": "REPEAT_OFF",
  "requestId": 202,
  "sessionId": "81bdf716-f28a-485b-8dc3-ac4881346f79",
  "startIndex": 0,
  "type": "QUEUE_LOAD"
}

Here I spent a long time trying without any customData, and just using the appId and contentId. Initially it seemed to work!

However, it turned out it only worked if the DRTV Chromecast app was already launched from another device. If launched directly from pychromecast the app would load, show a spinner, and then go back to idle. Here much frustration was spent; I guess the customData is actually needed. And indeed, putting that in works! But where do these tokens come from, and how do we get those tokens from Python?

Using Chrome’s developer tools (F12) on the DRTV page, and then searching globally (CTRL-SHIFT-f) for various terms (“expirationDate”, “customData”, “profileToken”, “accountToken” etc.) revealed some interesting code, that was as semi-readable as any pretty-printed minifyed Javascript. Eventually I found the tokens in local storage:

Using these tokens work really well, and allows starting playback!

Some further exploration proceeded: using the showDebugOverlay flag reveals that the DRTV player is just a rebranded Shaka Player. The autoplay functionality can be disabled by setting chainPlayCountdown to -1, which is honestly a real oversight that it cannot be disabled officially, to not have to rush to stop the playback of the item before the next autoplays.

With all the puzzle pieces ready, I prepared a pull request (still open) to add support for DRTV to pychromecast.

Fantus-button part 2 will follow, detailing the hardware build and network integration with the support from pychromecast.