So, for the Cambridge Raspberry Jam I wanted to get the EEG reader to be used for something. I decided it had to be a single signal and immediately Flappy Birds came to mind.
Nice simple one button game.
Decision made I went about coding the game in Python on the Raspberry Pi. I had already done the work to get the EEG
To hack the Mindflex and get it working with an Arduino I used this great guide and the code from the following article: http://frontiernerds.com/brain-hack.
This articles gives a lot of detailed background on the Mindflex and how it as well as lovely clear pictures showing where the TX and GND wires need to be attached.
For strain relief the article uses hot glue. I wrapped the wires around the already in place knot so the wires could not be pulled no matter what.
The Processing code provided didn't work for me on the Raspberry Pi and that's the main reason I had to code up my own visualisation program.
The example I used to send the data is BrainSerialTest with one small modification. I commented out the following line as it caused the import and breaking out of the variable in Python to fail. (Limit of my coding ability showing):
Serial.println(brain.readErrors());
To read this from Python I had to install pyserial as it wasn't installed by default using the following commands.
apt-get update
apt-get install python-serial
You will have to use sudo for this. I left it out from the commands so that you have to make the decision to run the commands
With the Mindflex hacked, and hooked to the Arduino and then sending it's data over serial I did the code to visualise the data. Here is the Python code I used on the Raspberry Pi for the last Egham Raspberry Jam
import serial
import pygame
# initialise pygame
pygame.init()
# Define the colors we will use in RGB format
black = [ 0, 0, 0]
white = [255,255,255]
red = [255, 0, 0]
blue = [0,0,255]
darkBlue = [0,0,128]
pink = [255,200,255]
green = [0,255,0]
orange = [255,102,0]
brown = [153,102,0]
yellow = [255,255,0]
purple = [128,0,128]
# Set the font for the text. Windows computer so usd Ariel
myfont = pygame.font.SysFont("OpenDyslexic", 18)
# Set the height and width of the screen
size=[800,600]
screen=pygame.display.set_mode(size)
# Fill the screen White
screen.fill(white)
# Put something in the application Bar
pygame.display.set_caption("Serial Mind reader")
pygame.display.update()
done = False
div = 2000
fatt = 20
fmed = 20
fdelta = 20
ftheta = 20
flalpha = 20
fhalpha = 20
flbeta = 20
fhbeta = 20
flgamma = 20
fhgamma = 20
ser = serial.Serial('/dev/ttyUSB0',9600)
# Clear the serial input buffer
ser.flushInput()
pygame.draw.rect(screen, white, (0,0,800,600), 0)
pygame.display.update()
while done == False:
for event in pygame.event.get():
if event.type == pygame.QUIT:
done=True
data = ser.readline()
signal,att,med,delta,theta,lalpha,halpha,lbeta,hbeta,lgamma,hgamma = data.split(",")
print signal
print att
print data
# clear screen before putting new square on it
pygame.draw.rect(screen, white, (0,0,800,600), 0)
pygame.display.update()
fatt = int(att)
fmed = int(med)
fdelta = int(delta)
ftheta = int(theta)
flalpha = int(lalpha)
fhalpha = int(halpha)
flbeta = int(lbeta)
fhbeta = int(hbeta)
flgamma = int(lgamma)
fhgamma = int(hgamma)
pygame.draw.rect(screen, red, (1,600-(fatt*4),75,fatt*4), 0)
pygame.draw.rect(screen, blue, (76,600-(fmed*4),75,fmed*4), 0)
pygame.draw.rect(screen, pink, (152,600-(fdelta/div/2),75,(fdelta/div/2)), 0)
pygame.draw.rect(screen, green, (227,600-(ftheta/div),75,(ftheta/div)), 0)
pygame.draw.rect(screen, orange, (303,600-(flalpha/div),75,(flalpha/div)), 0)
pygame.draw.rect(screen, darkBlue, (379,600-(fhalpha/div),75,(fhalpha/div)), 0)
pygame.draw.rect(screen, brown, (455,600-(flbeta/div),75,(flbeta/div)), 0)
pygame.draw.rect(screen, black, (531,600-(fhbeta/div),75,(fhbeta/div)), 0)
pygame.draw.rect(screen, yellow, (607,600-(flgamma/div),75,(flgamma/div)), 0)
pygame.draw.rect(screen, purple, (683,600-(fhgamma/div),75,(fhgamma/div)), 0)
screen.blit(myfont.render("Attention",1,black), (4, 1))
screen.blit(myfont.render("Mediation",1,black), (79, 1))
screen.blit(myfont.render("Delta",1,black), (155, 1))
screen.blit(myfont.render("Theta",1,black), (230, 1))
screen.blit(myfont.render("Low Alpha",1,black), (306, 1))
screen.blit(myfont.render("High Alpha",1,black), (382, 1))
screen.blit(myfont.render("Low Beta",1,black), (458, 1))
screen.blit(myfont.render("High Beta",1,black), (534, 1))
screen.blit(myfont.render("Low Gamma",1,black), (610, 1))
screen.blit(myfont.render("High Gamma",1,black), (686, 1))
# show the whole thing
pygame.display.update()
pygame.quit ()
As a side note. I did a D&T day at my kids school where I used this code for a Year 1 demonstration. All the kids got to see their brainwaves bouncing around as they moves, did maths and recited poems. It even went really flat when the kids closed their eyes and calmed down. Some of the kids had to wait so long that they missed their morning break for the opportunity to see their brains waves bouncing around,
Next onto getting the Flappy Birds inspired game (for the purest you could say helicopter inspired) game working.
I thought myself to code in the 80's on a Commodore 64 and to be honest I haven't updated my skills since, so doing the same in Python was a new challenge for me.
Fortunately, I read an article in Linux Format Issue 112 from December 2008 that made a top down racer using Python and Pygame in 100 lines of code. http://www.linuxformat.com/includes/download.php?PDF=LXF112.tut_code.pdf
Yes, using a tutorial from over 7 years ago. I may not be great at coding but I do hoard useful bits of knowledge. I bought the Mindflex from eBay in September 2014 and did nothing with it until January. Another bit of my electronics stash.
The Racer game if you think about it is the same but more complex than the Flappy Brain game with a few differences.
Racer scrolls vertically, Flappy Brain scrolls horizontally. That's just changing the value of the X variable instead of the Y variable
Then it's just change some of the graphics.
Once I figured out the Racer code I worked out the bits I could throw away and then bits I had to keep as well as worked out how to modify the code to do what I needed.
This took me a while to work through again due to a lack of knowledge or practice.
In the end I created the final game that was shown at the Cambridge Raspberry Jam.
Code Below:
from pygame import *
import random
import time
import serial
# set up the serial port
ser = serial.Serial('/dev/ttyUSB0',9600)
# Clear the serial input buffer
ser.flushInput()
# variables for colours
black = [ 0, 0, 0]
white = [255,255,255]
red = [255, 0, 0]
blue = [0,0,255]
darkBlue = [0,0,128]
pink = [255,200,255]
green = [0,255,0]
orange = [255,102,0]
brown = [153,102,0]
yellow = [255,255,0]
purple = [128,0,128]
# gap in wall
gap = 200
# width and height of screen
width = 1000
height = 600
count_gap = int(height/gap)
# 0 = hard, 1 = easier, 2 is easiest
difficulty = 2
# class to create sprites and render them
class Sprite:
def __init__(self,xpos,ypos,filename):
self.x = xpos
self.y = ypos
self.bitmap = image.load(filename)
def render(self):
screen.blit(self.bitmap,(self.x,self.y))
# screen size
size=[width,height]
# initialise pygame
init()
# create the screen (window)
screen = display.set_mode(size)
# Caption to go at the top of the window
display.set_caption("Flappy EEG 2")
# set the music, its volume and start to play. -1 means infinite loop
mixer.music.load("Pinky_and_the_Brain.ogg")
mixer.music.set_volume(0.5)
mixer.music.play(-1)
# crreate sound for crash
crasheffect = mixer.Sound("ouch.ogg")
# fill the screen with blue, update the screen
# not really required but I like it
screen.fill(blue)
display.update()
#time.sleep(1)
# create the sprites for the brain, columns and the background image
playerbrain = Sprite(20,200,"brain_75.png")
column1 = Sprite(1200,100,"column1.png")
column2 = Sprite(1200,100,"column2.png")
background = Sprite(0,0,"background.png")
# set fonts for different purposes
scorefont = font.Font(None,60)
datafont = font.Font(None,20)
overfont = font.Font(None,100)
# set default values for some variables
score = 0
maxscore = 0
quit = 0
gameover = 0
# master loop for the program. If quit == 0 then exit program
while quit == 0:
# flush the serial port of all data to begin fresh
ser.flushInput()
gameover = 0
# set the height of top column
column1.y = (random.randrange(0,(count_gap))*gap)-800
# set the height of the bottom column
column2.y = column1.y + 800 + gap
# start of loop (using while) to move the columns
# start off screen to the right
x = width +50
# x>-100 means still valid (yes there is a minus in there)
# gameover when collision
# quit selected. either pressed q or click x on window
while x >-100 and gameover == 0 and quit == 0:
# increment the score and if higher than maxscore make maxscore = score
score = score + 1
if score > maxscore:
maxscore = score
# update columns location and set x positions
x = x - 50
column1.x = x
column2.x =x
data = ser.readline()
# print data
signal,att,med,delta,theta,lalpha,halpha,lbeta,hbeta,lgamma,hgamma = data.split(",")
print "signal: " + signal
print "attention: " + att
print "data: " + data
intatt = int(att)
if intatt > 90:
intatt = 90
brainpos = intatt * 6
# set brain location based att (attention)
# is intatt near the gap above
if brainpos < column1.y +800 and brainpos > column1.y + 800 - (difficulty * 10):
playerbrain.y = column1.y +800 +70
print "brain near top and moved down!"
# is intatt near gap bottom
elif brainpos > column2.y-75 and brainpos < column2.y + (difficulty * 10):
playerbrain.y = column1.y +800 +70
print "brain near bottom and moved up!"
else:
playerbrain.y = brainpos
print "brain where is should be"
# print playerbrain.y
background.render()
playerbrain.render()
column1.render()
column2.render()
# display some information on screen
screen.blit(scorefont.render("Score: "+ str(score),1,white), (100, 5))
screen.blit(scorefont.render("High Score: "+ str(maxscore),1,white), (400, 5))
screen.blit(datafont.render("signal: "+ signal,1,white), (5, 570))
screen.blit(datafont.render("attention: "+ att,1,white), (150, 570))
screen.blit(datafont.render("playerbrain.y: "+ str(brainpos),1,white), (250, 570))
screen.blit(datafont.render("column1.y: "+ str(column1.y+800),1,white), (500, 570))
screen.blit(datafont.render("difficulty: "+ str(difficulty),1,white), (650, 570))
display.update()
# print playerbrain.y
# collision dection
if ((playerbrain.y < column1.y+801 or playerbrain.y > column2.y-74) and (x <150 and x > 20)):
mixer.music.stop()
mixer.Sound.play(crasheffect)
print "BUMP"
gameover = 1
# check if clicked x on window to exit
for ourevent in event.get():
if ourevent.type == QUIT:
quit = 1
# has key been pressed. K_q is to quit
if ourevent.type == KEYDOWN:
if ourevent.key == K_DOWN:
playerbrain.y = playerbrain.y+10
if ourevent.key == K_UP:
playerbrain.y = playerbrain.y-10
if ourevent.key == K_q:
quit = 1
# if game over show message
while gameover == 1 and quit == 0:
screen.blit(overfont.render("GAME OVER",1,yellow), (380, 260))
display.update()
# then wait for a key to pressed before starting again
for ourevent in event.get():
if ourevent.type == KEYDOWN:
if ourevent.key == K_0:
difficulty = 0
score = 0
gameover = 0
mixer.music.play(-1)
if ourevent.key == K_1:
difficulty = 1
score = 0
gameover = 0
mixer.music.play(-1)
if ourevent.key == K_2:
difficulty = 2
score = 0
gameover = 0
mixer.music.play(-1)
if ourevent.key == K_SPACE:
score = 0
gameover = 0
mixer.music.play(-1)
if ourevent.key == K_q:
quit = 1
score = 0
gameover = 0
The sharp eyed amongst you will recognise the bits taken from the Racer game and the bits taken from the original visualisation code I create.
I added the ability to Quit the game by pressing 'q' and also 3 difficulty levels. On the Game Over screen press 1, 2, or 3 with the higher number being easier.
It does this by widening the readings that will get you through the gap.
At the Cambridge Raspberry Jam I had the pleasure of meeting Alex Eames who brought us to HDMIPi and also the RasPi IO Duino which was another successful Kickstarter and is now for general sale. Alex did a great video which you can see below.
Following this the project was picked up by the Raspberry Pi Foundation and featured on their Blog (https://www.raspberrypi.org/flappy-brain/) and most recently was featured as The MagPi Magazine Issue 35 in their Amazing Pi Projects List (https://www.raspberrypi.org/magpi/)
The MagPi magazine is a a resource for everybody with a Raspberry Pi. It includes articles, project details and tutorials in everything from Scratch, to Python and C++ and has software and hardware projects. Also, they announced recently that it is going to become a physical magazine soon and be available to buy in shops as well as the current PDF download and acces through the Apple App Store and Google Play stores.
It's been a fun few months and to me is an example of what can be done if you take advantage of the resources out there if you have idea. For this project I used some great code and guide to hack the Mindflex. Code from a 2008 Linux Format article and some previous work I did to figure out how to read serial data on the Raspberry Pi.
I will be bringing Flappy Brain to the next Egham Raspberry Jam. It is on the 12th of July 2015 and as always is free to attend. All that's required is to register beforehand on the Eventbrite page.
http://eghamjam8.eventbrite.com
Maybe I'll see you there.
