Category Archives: Sometimes, I am creative

Fish Plays Piano falls short – Twitch Plays Chess instead!

Sometimes I get this little voice inside me, that wants to do something completely different and just ignore all the things I have to do and take care of. Apparently this time it was to make a fish play music! (Yes, I am an adult!)

This weekend I did not get to record for my YouTube channel or do anything else that you could call “useful”. Because I was playing around with some concept for a new Twitch channel. My first try to put my fish Stumpy in front of the camera and let him play the piano, worked after roughly 12 hours of work. But due to their inability to play beautiful melodies and lay around a lot of the time, kinda turned out to be … not that interesting. I’d much rather watch a human play the piano on stream 😀

The implementation was surprisingly easy with the help of MATLAB’s image processing features and Arindam’s blog post / template I could work with. I did some minor tweaks to the script, mainly in the form of dividing the screen into a grid and checking the coordinates from the blob with said grid and then sending keystrokes to a online piano tool.

After I streamed this for a while, I decided I would try out something along the lines of “Twitch Plays Pokemon”, just with a different game: Battle Chess! I spent a few hours trying to set up the system with some Python scripts, with no luck (Getting Python & PyWin32 to run on my system was a headache).

I then looked a bit further and found an open source tool called OpenTwitchPlay. The implementation was fairly easy and the stream was up and running in half an hour and seems now to run stable. I will have a closer look at the source code and see if I can modify it to fit it the channel; other chess software is also something I am considering, since this would maybe allow more complex commands. Luckily I have a dedicated system for the stream (It’s my German notebook, that I won’t be able to sell here heh)

The Stream is currently running for 20+ hours and so far Twitch lost 2 times against the (slightly slow) NES AI. Currently it picks up in pace and some dedicated viewers are taking on the challenge of beating the AI. It certainly is going to be tricky, when more player are going to join the stream, but so far it’s been a blast watching them and listening to Nectarine Demoscene Radio. (Haha, please ignore the April Fools Bieber theme)

Chess is very much like Starcraft 2 for me: I enjoy playing it once in a while, but I don’t have the perseverance to get really, really good at it; Since I love tinkering on stuff like this, I am looking forward to improving TwitchLovesGames!