Overview

Sous Chef is a comprehensive companion app for hosting cooking shows on the Twitch livestreaming platform. It integrates with every aspect of the streaming experience, providing animated video overlays for rendering show graphics, a web-based control panel for switching cameras and managing other show elements, and integration with Twitch's chat functionality for engaging with audiences.

The application is built upon a variety of open source tools, including Electron, Express, Socket.io, React, Webpack, and Bootstrap. Sous Chef has a modular architecture, allowing pieces to be added or removed depending on the host's particular needs. It was developed in collaboration with @AndrewCooksLive, and its code can be found on GitHub. Icons and graphics were provided by the Mutant Standard alternative emoji set.

Features

• Runs locally as a native application on both macOS and Windows.
• Provides a web-based control panel that can be accessed from any computer, tablet, or smartphone on the host computer's LAN.
• Includes integration with OBS-Websocket plugin for software-based scene switching and device muting.
• Locally hosts a web overlay which can be integrated into the Open Broadcaster Software (OBS) or Streamlabs OBS. Overlay elements include:
  • Video inlay panels
  • Stream info panels (Recipe details, etc.)
  • Multiple simultaneous timers
  • Image and Tweet embeds (including animated gifs and Twitter videos.)
  • Probe thermometer temperature charts and live readouts
  • iTunes integration for displaying background music details.
  • Full-screen confetti and other celebratory animations
• Supports Twitch's chat API. Can post messages to chat, respond to viewer commands, allow select viewers to trigger overlay actions, and engage viewers with automated trivia questions.
• Additionally supports Twitch's event APIs, allowing actions to automatically trigger when users perform standard actions such as following or subscribing.

Background

This project might take a bit more explaining than usual. I never saw myself as the type of person to co-host a cooking show, but 2020 turned out to be an interesting time for everyone. By now most people are at least passingly familiar with Amazon's livestreaming video platform Twitch. It originally launched as a platform solely dedicated to allowing gamers to livestream their video gaming sessions, but in the years since it launched, they've opened it up to all sorts of non-gaming topics.

Twitch streams now feature everything from artists painting to live music concerts, or in our case, a homegrown cooking show. Hosting a show on Twitch is a bit like running you own TV show, with a few subtile differences. Most notably, there's a chatroom running alongside every broadcast. Your typical Twitch streamer isn't broadcasting out into the void, but rather interacting with a community of regular viewers. For our stream in particular, @AndrewCooksLive handles the bulk of the culinary workload, and I help produce things. (I've been known to occasionally hop in and chop some onions when things get hectic.) It's been a fun way of staying connected to other people during these quarantine times, and stopped us from falling into a culinary rut.

In addition to having to come up with something new to cook every week, it turns out there is a ton of specialized technology that goes into producing a livestream. Your basic video game stream simply needs a webcam, a microphone, and streaming screen capture software. Your basic cooking stream however typically involves two or three cameras, wireless microphones, streaming software, and usually some additional plugins or programs. That's where Sous Chef comes in. What originally started as a side project to show recipe info and egg timers on our stream has morphed into the glue holding the entire show together.

Our Setup

While the software's important, I want to touch briefly on the hardware we're working with. Our setup and configurations could be its own specific writeup, but in brief, we're currently using:

• Three Logitech C920 Webcams
• A wireless IP camera
• Two wireless UHF lavalier microphones
• A PC with enough horsepower to handle all of the above
• A kitchen's worth of pots, pans, and accoutrement

Everything's arranged in the kitchen through a variety of camera arms, tripods, and 3D printed mounts. (It's seriously tricky to hang a camera over a cutting board while still being able to chop efficiently.) Our next big project is to improve the lighting of the show, but for now, it's a pretty solid cooking stream setup for only a few hundred dollars. It also helps that we're currently not entertaining company who might have more than a few things to say about the number of USB cables scattered about our kitchen.

On the software side of things, you've got one big choice you need to make: Are you going to go with the Open Broadcaster Software's OBS Studio, or Streamlabs.com's custom Streamlabs OBS? OBS Studio is the classic piece of open source software that helped jumpstart the particular streaming revolution that we're currently in, while Streamlabs OBS is a proprietary layer of niceties on top of the open source software to make things run a bit more smoother. For the purposes of Sous Chef, either work. You simply need to be able to create a web browser overlay element, which both support.

To touch on Streamlabs.com for a moment: Their site touts it as the "All-in-One Live Streaming Software". If this sounds familiar, it's because Sous Chef is essentially a competitor to Streamlabs. Streamlabs can handle onscreen elements, manage chatbots, and even take care of complicated processes like taking money for donation drives. It's essentially the Wordpress of web streaming, if you have any experience in the blog CMS world. Streamlabs.com is the Swiss army knife of live streaming services, while Sous Chef is designed to be more specifically focused on the elements of live streaming that would be useful during a cooking stream. I generally prefer working with OBS Studio to avoid Streamlabs' constant nudging that I pay for a premium subscription, or check out their gallery of paid themes, but I still will occasionally use some of Streamlabs.com's different plugins for non-standard parts of our show. Either way, whichever flavor of OBS you decide to go with, Sous Chef should have something to offer.

Getting Started With Sous Chef

Sous Chef is structured a bit unusually as far as Electron applications go. Electron is a NodeJS framework which allows you to create cross-platform desktop applications using standard web technologies like HTML, CSS, and Javascript. In its most simple form, Electron creates a desktop app window, and then wants you to point it at an HTML file, a localhost URL, or an external web URL.

If you'd like to do something a bit more complicated than serving up a simple index.html file, there are two different approaches. The first is to use the popular Create React App boilerplate project to develop a complex React frontend for your Electron application. The second approach is to create a localhost environment using the Express framework to create a robust backend that's also capable of serving the Electron frontend.

Sous Chef takes advantage of both of these approaches. Express manages a backend API, and serves the Sous Chef control panel to any browser on the local network. Create React App handles the inner workings of the control panel itself, and brings along some niceties for developing a complex React application like module hot-reloading. When working in dev mode, Create React App will host the frontend on its own localhost port using webpack, and Electron and Express will both just reference this instance. When compiling the application, Create React App will compile the application down to an index.html file that the Express instance manages serving.

So at a high level, the application structure looks something like this:

• sous-chef/ (Top-level app folder)
  • plugins/ (Express back-end modules)
    • recipe.js
    • timer.js
    • ...
  • public/ (Create React App flat files)
    • favicon.ico
    • index.html
    • ...
  • src/ (Create React App React modules)
    • index.css
    • index.js
    • Control/
      • Control.js
    • Overlay/
      • Overlay.js
    • ...
  • main.js (App entry point, manages Electron & Express configs)
  • package.json (App configs)

Go ahead and run npm install. While npm fetches all the required packages, go ahead and take a look at the package.json file's script section. At the risk of losing you to another bulleted list, you can see the various commands that will finally allow us to finally get this bad boy up and running.

• "start": "cross-env PORT=3000 react-scripts start"
  • Launch the application's Create React App
• "electron": "wait-on tcp:3000 && electron-forge start"
  • Launch the application's Electron main.js component only after it sees a service running on port 3000.
• "dev": "concurrently -k "BROWSER=none npm start" "npm:electron""
  • Use concurrently to launch the start and electron scripts at the same time. Supress Create React App's standard browser-opening behavior.
• "build": "cross-env PORT=80 react-scripts build"
  • Build the Create React App instance using port 80.
• "make": "cross-env PORT=80 react-scripts build && electron-forge make"
  • Build the Electron application using port 80.
• "package": "cross-env PORT=80 react-scripts build && electron-forge package"
  • Build the Create React App instance using port 80, then make and package the Electron application for distribution.

As you can see, these scripts typically build on top of each other. npm run dev launches npm start and npm electron together, while npm run package kicks off npm run build and electron-forge package at the same time. Along with the packages I've already discussed, you've probably noticed that Sous Chef takes advantage of the Electron Forge toolkit to streamline the build and distribution process.

You're almost ready to get started. There's just one last piece that needs taking care of. Sous Chef uses the Twitch and Twitter  APIs for fetching data and interacting with 3rd party services. You'll need developer API keys for both websites, so head over to their developer sections, register, and create a new dev application. Once you've got Twitch's client ID, go ahead and copy the format in .twitch-keys_example.json to create .twitch-keys.json. Paste your client ID into the new file's json. Do the same for .twitter-key_example.json. You'll need to put your Twitter consumer key, consumer secret, and bearer token into .twitter-key.json. Twitter can take a few days to create a dev account for new users, so go ahead and do this process now if you're thinking about using Sous Chef.

Alright, that's everything! Go ahead and fire up npm run dev, and you should see Sous Chef start. If everything looks good, you can use npm run package to create a nice compiled application with a single icon to click.

Using Sous Chef

Basic Setup

Once you've got OBS up and running, you'll need to integrate its overlay into your OBS scenes. Create a browser overlay element, and make sure it's the same resolution and size as your screen. Set the source to Sous Chef's overlay URL. There's a few options for this, which can be found in the Config tab of Sous Chef. There's the human-friendly http://souschef.local/overlay for system's with Apple's Bonjour multicast DNS service installed, http://[Your IP]/overlay as a fallback for systems without Bonjour, and http://localhost/overlay if you don't need to access Sous Chef on your LAN. If Sous Chef is running in dev mode, make sure to put :3000 after the domain.

If you'd like to create a scene without the video inset panel, add the query string parameter ?noInset=true to the end of the URL.

If you haven't already, go install the OBS-Websocket plugin. This will allow Sous Chef to remotely control OBS from outside of the OBS application. You may keep OBS-Websocket's settings at their defaults. If you need to change the default port or would like to add authentication, these settings are managed in /plugins/obs.js via the obs-websocket-js package.

The "Now Playing" iTunes integration does not require any specific configuration. If you're using iTunes on Windows, iTunes on the Mac, or Apple's new Music app on the Mac, Sous Chef will automatically show song information and cover art when a new track starts playing.

The Sous Chef control panel is divided into three main sections: Cook, Prep, and Config.

Cook Tab

Scene Switching & Hotkeys

The top of Sous Chef has your classic scene-switcher interface. The button layout is designed to mimic Dave Madison DIY Streamdeck project. We've got one at the base of the stove, so it's nice to have the same button layout mapped in software as well. It provides a quick interface to move between the stove, cutting board, and kitchen cameras, as well as quick actions such as muting the microphones or toggling confetti.

Active Timer

The Active Timer panel lists the timers currently being displayed on the overlay. Once a timer expires, the control panel will play a whistle sound effect, and the expired time will begin to flash red. There will also be an attention-grabbing red flashing boarder on the overlay itself. Timers can be canceled or restarted at any time. The scene Switching and Active Timers panels are responsive, so they'll be displayed side-by-side if the control panel is large enough.

New Timer

The New Timer panel provides an interface for creating new timers. Users can optionally give the timer a unique name, and either provide a specific amount of minutes and seconds, or use one of the provided hotkeys for common timer amounts.

Show Image

Sometimes a a picture is worth a thousand words, and you'd just like to show your viewers a specific image from somewhere on the web. The Show Image panel allows you to create a pop-up image panel on the overlay software. It accepts an image URL, and provides options for rotating images in the case that OBS's browser element has difficulty reading a camera's rotation EXIF data. The control panel provides a small thumbnail of the currently displayed image so that the producer can have immediate feedback as to whether or not anything is currently being displayed on the overlay.

Show Tweet

Similar to the Show Image panel, the Show Tweet panel allows you to embed a tweet from Twitter.com on the overlay. The embed panel supports Twitter video and Twitter's .gifv videos, allowing you to quickly show off some animated content.  

Probe Thermometer Control

This section could honestly be a standalone blog post. After a few streams involving frying, we realized that there's not much for the audience to watch while the oil warms up. There's a number of fry recipes that rely on maintaining a particular fry oil temperature, so we decided why not feed two birds with one scone and display our probe thermometer's output on the overlay itself?

This is of course easier said than done. We'd been using the reliable Thermoworks Chef Alarm, but it doesn't provide any Bluetooth or Wifi connectivity that we could piggyback off of for the overlay. Rather then investing in a new thermometer, I decided to build a small probe-thermometer-to-PC adapter using an Arduino-based microcontroller. The Arduino reads the electrical resistance of the probe thermometer, passes it through the Steinhart–Hart equation to determine how hot the probe is, and then sends the data to the PC with a basic USB serial signal. There's even a small seven-segment display built in for some immediate feedback.

The Probe Control panel in Sous Chef lets you control what you'd like to do with this thermometer data. Once you selected the proper serial port, you can either display the current temperature, or you can show a live-updating plot of the data with a Chart.js graph. There's also the ability to set an alarm to go off when a target temperature is reached.

Toggles

Alright, this one's a bit more straightforward. There's a few less-frequently used overlay elements such as confetti or an announcement banner. This panel does what it says on the tin, and allows you to toggle these on and off.

Remote Producers

Part of streaming is letting your audience get in on the fun. A few of the overlay's elements have been tied to chat commands like !cuttingboard or !confetti. The Remote Producers panel allows you to grant permissions to specific users to take over producing responsibilities for the stream.

Prep Tab

Recipe & Drink

The Recipe and Drink panels control a small overlay element which let people know what's on the menu for the evening, and what cooking beer you're enjoying during the stream. The URL section controls a small chat bot which responds to !recipe or !drink.

Reminders Bot

The Reminders Bot is a Twitch chatbot which posts specific messages at a regular interval. This is usually used to remind people of ongoing charity drives and other events.

Trivia Bot

We've been experimenting with a Trivia Bot recently as a way to boost viewer engagement. It's currently managed through a small trivia.json file stored within Sous Chef itself. The trivia bot will post a message at a regular interval, allowing people to score points for various stream prizes.

Config Tab

Twitch Integration

The aforementioned Twitch integration is controlled through Twitch's OAuth API. Sous Chef requires two separate logins. Collecting stream event data requires the broadcaster to log in, while posting chat bot messages requires the specific chat bot account to login.

OBS Integration

The camera switching components rely on OBS's websocket plugin. The connection kicks in automatically when Sous Chef launches, but its status can be double-checked here if something's gone wrong.

Sous Chef Information

The application has two main web endpoints: The control panel interface, and the overlay itself. The Sous Chef Information panel provides links to these pages in a variety of forms for testing purposes. The most user-friendly of these links is the souschef.local Bonjour networking (mDNS) URL. If the client machine does not have Bonjour installed however, a direct IP option may be used as well.

Closing Thoughts

Sous Chef has been a fun quarantine project. It's helped me learn a lot about NodeJs, React, and integrating with 3rd party services and clients. The app is currently fine-tuned to our cooking stream's specific needs, but if it looks like something you would be interested in using, don't hesitate to reach out to me with ideas. All of the code is up on my Github for forking as well.