Thursday, February 16, 2017

Green Screen photo booth using Raspberry Pi

We've seen a number of photo booths herehere and even the All Seeing Pi being done with Raspberry Pi. All of these photo booths are based on taking a picture and maybe putting an overlay to add a banner or something like a funny hat or mustache.

My thoughts were could the Raspberry Pi 3 do Green Screen.  You know that special effect from the movies where the background is removed and a different image put in it's place.

As a kid I think the first time I remember seeing this was Superman.

How cool would that be to make your own version of the special effects used in big budget movies.

So, I started the search for tools under Linux that would permit the green screen to be done.
As a short summary green screen/chroma key is where a a single colour is removed from an image.
Most often it is green as modern cameras are more sensitive to green and a bright green works best as it's less likely to be a colour in a natural scene.  I remember when I was younger hearing it being done with blue screen as well.  

After a bit of looking I found Imagemagick a jack of all trades image processing tool.  It has a function to remove a single colour from an image and as importantly for my use includes a 'fuzzy' search for the colour which gave a bit of tolerance to the lighting.  

In the code the most important line is:
os.system('/usr/bin/convert -limit thread 4 ' + folder +'imagecam.png -fuzz '+fuzzpercent+' -transparent "#'+rgbnum +' " ' + folder + 'imagecamt.png')
I know there is a lot going on in there.  She short version is it takes imagescam.png, makes transparent the colour rgbnum with a tolerance of fuzzpercent and saves it as imagecamt.png

This forum thread was really useful in understanding how to use it.

Once I had my head around this it was then a matter of doing the rest of the code.
Take the picture
Remove the green
Layer a background and the image with the green removed. 
For testing I used some of my kids PlayMobil and a piece of A0 green card.  

All looks like it's working well even with the ability to change the background using the arrow keys

Now for the next level.
Scale it Up to Life Size !!!!

This of course means I needed a green screen background with standard that I bought from eBay.  No idea when I'll use the black or  white backgrounds that came with this kit, but I have them now.

And then Tweet the pictures.
For Tweeting I used tweepy and followed the excellent guide by Alex Eames @RasPiTV 

Finally, wouldn't it be great to have a little remote control and not have to rely on a keyboard.  Since the Raspberry Pi 3 has Bluetooth I thought this might be the ideal solution.  No wires and no messing about.  Since Pygame was already being used for the displaying of the images and I knew Pygame had joystick support built in this looked like the obvious choice.

Again, on eBay I found this small little Bluetooth gamepad and thought it would be perfect.  Super small which means you can have it in your hand but not interfere with your final picture.

Iddy biddy, teeny weeny, black gamepad 

My usual style is before bring a new feature into a project I like to test it standalone to make sure it works.  For the gamepad I created a small Python/Pygame program to test the gamepad 

It paired with the Raspberry Pi first time and worked perfectly with the gamepad test program.  So, 100% sure it will function with the chromaCam setup.

Now all the parts are in place.  PiCamera to take the picture.  Imagemagick to remove the background colour.  Pygame to merge background, foreground picture with green removed and finally an overlay with the background images changed with the keyboard or the super small gamepad.  (Oh yeah, the circular thing at the bottom is an analogue joystick)

The chromaCam set up got it's outing at the last Wimbledon Raspberry Jam.  Here are some pictures.
On the day I used a camera tripod and a lot of tie wraps to hold the Pi Display and the Pi Camera onto the tripod. I would definitely recommend a more secure mounting method 

Getting set up

Who doesn't love a bit of Harry Potter -

Those Lions are dangerous

That doesn't look like a sensible thing to do -

Final notes.
If you did click the link above that explains Chroma Key/Green Screen then you'll have read that lighting is really important.  The goal is to one specific colour. If your lighting is uneven then due to shadows or folds in the green screen the green will be different giving varying results.
Taking out too much -

I'm sure the top of the Raspberry Pi logo isn't transparent

A few bits of rogue green -

If you want to make your own Green Screen Photo Booth then the code is available on GitHub

Final Thoughts:
Yes, this project worked and was great fun to build and see people using it.
For the project I limited the image size to 640x480 as even at that resolution it took about 2 seconds for the picture to be updated.  This meant it took a little bit of patients to get the picture you wanted before pressing the button to Tweet the image.

I took the set up to Hack Horsham as well but I didn't bring my own lighting and the lighting in the room was perfect for a Jam but didn't give enough contrast for chromaCam to work.  So, it didn't make the final cut and I stuck to Button Flash.

You can see the setup in the mannequin challenge video tweet. 

Again, this was a great learning experience for me and once the computing power for the Raspberry Pi enables it or access to the GPU is supported this will be amazing when the green screen transparency can be done in real time.  
I expect as new devices come out I'll be revisiting this project to see if the performance is improved. 

Making Games with Scratch and Sense Hat on Raspberry Pi

I'm a big fan of the Raspberry Pi Sense Hat and Scratch for doing interesting things and learning to code.
Previously I made a Sense Hat Cave Game using Python where you tilt the Sense Hat to move around as well as a Crane controller programmed in Scratch.
This time the goal was to create game that uses the Sense Hat as the display while programmed in Scratch.  The best of both world.  The goal for me was to lean to control the pixels a bit better in Scratch.

The result is Collect the Dots a simple game where you're a blue dot collecting red dots.  It's kind of like snake without the tail.

You control your dot using either the small joystick on the Sense Hat or use the keyboard on the computer.  Press Down to start the game.  You'll see in the video it tells you this on the Sense Hat, but if no Sense hat is attached and you're running on screen there is no prompt.

As can be seen in the video the on screen display mirrors on the Sense Hat dots, so either can be used.
Strangely even though the screen display is bigger I found it easier to play if I only looked at the Sense Hat display.
The game can also be played without the Sense Hat. Uploaded to Scratch website

Embedded from the Scratch Website

You can download the code from GitHub

TIP: As the wonderful Cat Lamin points out in her blog post about using doing an animation on the Sense Hat using Scratch you have to run the code once to enable the sensors and capability of the Sense Hat.  So, the first time you run the program on a Raspberry Pi with a Sense Hat it may not work correctly.  Just stop it and start again and all will be perfect

A thought for an enhancement would be to control your dot by tilting the Sense Hat rather than using the joystick or keys.