Thursday, February 16, 2017

Green Screen photo booth using Raspberry Pi

We've seen a number of photo booths herehere and even the All Seeing Pi being done with Raspberry Pi. All of these photo booths are based on taking a picture and maybe putting an overlay to add a banner or something like a funny hat or mustache.

My thoughts were could the Raspberry Pi 3 do Green Screen.  You know that special effect from the movies where the background is removed and a different image put in it's place.

As a kid I think the first time I remember seeing this was Superman.


How cool would that be to make your own version of the special effects used in big budget movies.

So, I started the search for tools under Linux that would permit the green screen to be done.
As a short summary green screen/chroma key is where a a single colour is removed from an image.
Most often it is green as modern cameras are more sensitive to green and a bright green works best as it's less likely to be a colour in a natural scene.  I remember when I was younger hearing it being done with blue screen as well.  


After a bit of looking I found Imagemagick a jack off a trades image processing tool.  It has a function to remove a single colour from an image and as importantly for my use includes a 'fuzzy' search for the colour which gave a bit of tolerance to the lighting.  

In the code the most important line is:
os.system('/usr/bin/convert -limit thread 4 ' + folder +'imagecam.png -fuzz '+fuzzpercent+' -transparent "#'+rgbnum +' " ' + folder + 'imagecamt.png')
I know there is a lot going on in there.  She short version is it takes imagescam.png, makes transparent the colour rgbnum with a tolerance of fuzzpercent and saves it as imagecamt.png

This forum thread was really useful in understanding how to use it.

Once I had my head around this it was then a matter of doing the rest of the code.
Take the picture
Remove the green
Layer a background and the image with the green removed. 
For testing I used some of my kids PlayMobil and a piece of A0 green card.  


All looks like it's working well even with the ability to change the background using the arrow keys

Now for the next level.
Scale it Up to Life Size !!!!

This of course means I needed a green screen background with standard that I bought from eBay.  No idea when I'll use the black or  white backgrounds that came with this kit, but I have them now.

And then Tweet the pictures.
For Tweeting I used tweepy and followed the excellent guide by Alex Eames @RasPiTV 

Finally, wouldn't it be great to have a little remote control and not have to rely on a keyboard.  Since the Raspberry Pi 3 has Bluetooth I thought this might be the ideal solution.  No wires and no messing about.  Since Pygame was already being used for the displaying of the images and I knew Pygame had joystick support built in this looked like the obvious choice.

Again, on eBay I found this small little Bluetooth gamepad and thought it would be perfect.  Super small which means you can have it in your hand but not interfere with your final picture.

Iddy biddy, teeny weeny, black gamepad 

My usual style is before bring a new feature into a project I like to test it standalone to make sure it works.  For the gamepad I created a small Python/Pygame program to test the gamepad 

It paired with the Raspberry Pi first time and worked perfectly with the gamepad test program.  So, 100% sure it will function with the chromaCam setup.

Now all the parts are in place.  PiCamera to take the picture.  Imagemagick to remove the background colour.  Pygame to merge background, foreground picture with green removed and finally an overlay with the background images changed with the keyboard or the super small gamepad.  (Oh yeah, the circular thing at the bottom is an analogue joystick)

The chromaCam set up got it's outing at the last Wimbledon Raspberry Jam.  Here are some pictures.
On the day I used a camera tripod and a lot of tie wraps to hold the Pi Display and the Pi Camera onto the tripod. I would definitely recommend a more secure mounting method 

Getting set up https://twitter.com/MrTomsWorld

Who doesn't love a bit of Harry Potter - https://twitter.com/rdhayler


Those Lions are dangerous

That doesn't look like a sensible thing to do - https://twitter.com/HackHorsham

Final notes.
If you did click the link above that explains Chroma Key/Green Screen then you'll have read that lighting is really important.  The goal is to one specific colour. If your lighting is uneven then due to shadows or folds in the green screen the green will be different giving varying results.
Taking out too much - https://twitter.com/Codepope

I'm sure the top of the Raspberry Pi logo isn't transparent


A few bits of rogue green - https://twitter.com/gowolade

If you want to make your own Green Screen Photo Booth then the code is available on GitHub


Final Thoughts:
Yes, this project worked and was great fun to build and see people using it.
For the project I limited the image size to 640x480 as even at that resolution it took about 2 seconds for the picture to be updated.  This meant it took a little bit of patients to get the picture you wanted before pressing the button to Tweet the image.

I took the set up to Hack Horsham as well but I didn't bring my own lighting and the lighting in the room was perfect for a Jam but didn't give enough contrast for chromaCam to work.  So, it didn't make the final cut and I stuck to Button Flash.

You can see the setup in the mannequin challenge video tweet. 


Again, this was a great learning experience for me and once the computing power for the Raspberry Pi enables it or access to the GPU is supported this will be amazing when the green screen transparency can be done in real time.  
I expect as new devices come out I'll be revisiting this project to see if the performance is improved. 



Making Games with Scratch and Sense Hat on Raspberry Pi


I'm a big fan of the Raspberry Pi Sense Hat and Scratch for doing interesting things and learning to code.
Previously I made a Sense Hat Cave Game using Python where you tilt the Sense Hat to move around as well as a Crane controller programmed in Scratch.
This time the goal was to create game that uses the Sense Hat as the display while programmed in Scratch.  The best of both world.  The goal for me was to lean to control the pixels a bit better in Scratch.

The result is Collect the Dots a simple game where you're a blue dot collecting red dots.  It's kind of like snake without the tail.



You control your dot using either the small joystick on the Sense Hat or use the keyboard on the computer.  Press Down to start the game.  You'll see in the video it tells you this on the Sense Hat, but if no Sense hat is attached and you're running on screen there is no prompt.


As can be seen in the video the on screen display mirrors on the Sense Hat dots, so either can be used.
Strangely even though the screen display is bigger I found it easier to play if I only looked at the Sense Hat display.
The game can also be played without the Sense Hat. Uploaded to Scratch website https://scratch.mit.edu/projects/144708660/

Embedded from the Scratch Website



You can download the code from GitHub

TIP: As the wonderful Cat Lamin points out in her blog post about using doing an animation on the Sense Hat using Scratch you have to run the code once to enable the sensors and capability of the Sense Hat.  So, the first time you run the program on a Raspberry Pi with a Sense Hat it may not work correctly.  Just stop it and start again and all will be perfect

A thought for an enhancement would be to control your dot by tilting the Sense Hat rather than using the joystick or keys.

Monday, January 30, 2017

Controlling screen brightness on pi-topCEED on Raspbian Jessie with guizero app and keyboard shortcut

I backed the pi-topCEED during the crowd funding campaign and have used it a lot since.

pi-topCEED

My kids also use it for homework and general Minecraft playing.
As I like to make things I have a number of microSD which means I regularly use it with Raspbian Jessie.

There are a couple of drawback to using Raspbian on the pi-topCEED.  The pi-topHUB doesn't shutdown the power completely leaving the backlight on and the Raspberry Pi in kind of a suspecd mode. The red LED is still on.  Also, there is no way to control the screen brightness.

Doing the usual Google to see if there was a way around this I came across this GitHub repository that looked like it would do the job.  https://github.com/rricharz/pi-top-install
It includes code to shutdown the pi-topHUB which also completely shuts down power to the display and to the Pi as well as provide a program called 'brightness' to control the screen brightness.

After following the instructions in the README.md I had the pi-topHUB shutting down correctly and if from the terminal I type brightness increase the screen brightness would go up and vice versa brightness decrease did what you'd expect.

All very good, but not very user friendly if you're running the GUI which is where I spend most of my time. 

With that in mind and having recently heard good thing about guizero (https://github.com/lawsie/guizero) which is based on tkinter and is said to make building GUI apps  in Python easier.

It looked to me like a simple app with 2 buttons [Darker] [Brighter] is all I needed so I read up a little on guizero.  Using the App using grid layout example (https://lawsie.github.io/guizero/app/) I soon has a small app that did exactly what I wanted.
Code available on Git Hub (https://github.com/winkleink/pi-top-brightness)

The program itself is quite short. Code below.  This makes 2 buttons.  Button1 and Button2.  When you press them they run the relevant system comment to change the brightness. 


from guizero import *
from subprocess import call

def brighter():
    call(["brightness", "increase"])

    
def darker():
    call(["brightness", "decrease"])
    
    
app = App(title="pi-top", height=50, width=145, layout="grid", bgcolor=None)

button1 = PushButton(app, brighter, text="Brighter", grid=[0,1])
button2 = PushButton(app, darker, text="Darker", grid=[0,0])
app.display()


Here is a video showing it working.


After a quick chmod +x  pi-top-brightness-gui.py the program was executable and so I could run it directly.

Then I added the program to the main menu under the System Tools section.
In the video you can see I have a pi-top icon.  This I took from their Twitter account (https://twitter.com/GetPiTop)  I expect they'll be OK with that as I'm adding even more branding to my pi-topCEED.
I explained in a recent post on backing up Minecraft Worlds how to add a program to the menu (http://www.winkleink.com/2016/07/minecraft-pi-backup-worlds-on-exit.html)

It's great to have a little gui app to change the brightness of the screen and even better to learn a new skill with guizero.

One thing I didn't mention is that when I rebooted the Raspberry Pi the first time after installing the programs the screen brightness was far lower than I wanted it be by default.

To fix this I added a single line cronjob.  Crontab with cronjobs is a way to schedule when programs run automatically with one of the options being to run at boot up. It's a simple way to get something running each time you login. It's the system used by people who lets say want to check their network connection every hour.  Just setup a cronjob to run every hour and it will take care of it for you.   I used this previously to enable the Bluetooth keyboard and trackpad or the NexDock. (http://www.winkleink.com/2016/10/nexdock-thoughts.html)

So, I used crontab to set the brightness of the screen to 8 on startup.

In the terminal type:

crontab -e

If this is your first time it asks which editor you want to use.  I do [2] for nano.
Then the crontab file is opened where you put the jobs you want to run.
There are some comments in the file to help you get started, but if you want to do serious scheduled jobs then it's worth 

Go to the bottom of the file and type

@reboot brightness 8

Then the usual nano [CTRL]-[x] to save and close.  Answering [y] to save the file with the same name

Now when I start up the pi-topCEED the default brightness is '8' which suits me.


Finally, finally, the rricharz GitHub explains how to set the keyboard shortcuts for the pi-top laptop so the brightness keys work.  All good if I had a pi-top laptop, but I have a pi-topCEED where it's bring your own keyboard and the one I'm using doesn't have any fancy keys.
So, I looked up how to add keyboard shortcuts.  Again, it ends out to be not too hard.  Just added a couple of entries into a config file.

First thing I had to do is decide what key combination I wanted to use. I settled on:
[Shift]-[Windows Key]-[Up Arrow] for brighter
[Shift]-[Windows Key]-[Down Arrow] for darker

To do this you need to edit the lxde-pi-rc.xml file.
The file is a in a hidden folder in your home folder. If using the default user pi then the following command will open it 

nano /home/pi/.config/openbox/lxde-pi-rc.xml

As the file is in the home folder it doesn't need sudo to edit it

Then scroll down to the keyboard section and add the following to the end.
Easiest way is in nano find the </keyboard> text using the command [Ctrl]-[w] to do a Where is

Make sure you use spaces for the indent and get the <keybind> to line up with the other <keybind> entries

    <keybind key="S-W-Up">
      <action name="Execute">
        <command>brightness increase</command>
      </action>
    </keybind>
    <keybind key="S-W-Down">
      <action name="Execute">
        <command>brightness decrease</command>
      </action>
    </keybind>

To finish like above use [Ctr]-[x] to save and close.

To test select [shutdown] in the drop down menu and then [logout].
No need to reboot to enable the keyboard shortcuts.
Once you log back in the lxde-pi-rc.xml setting will be re-enabled and the 2 new key bindings will be active giving you the ability to change the brightness of your pi-top/pi-topCEED using [Shift]-[Windows Key]-[Up Arrow] and [Shift]-[Windows key]-[Down Arrow]

This has been a fun little project to improve the use of the pi-topCEED.
I like the work rricharz has done getting the pi-topHUB to turn off completely and to control the brightness as well as Laura Sach has done to make guizero so easy to use.

With the help of other peoples effort I now have the pi-topCEED completely shutdown as well as the Raspberry Pi and the ability to change the screen brightness using either a little gui app or with the keyboard.  Not a bad result for a couple of hours.

If you have any questions let me know in the comments.