My thoughts were could the Raspberry Pi 3 do Green Screen. You know that special effect from the movies where the background is removed and a different image put in it's place.
As a kid I think the first time I remember seeing this was Superman.
How cool would that be to make your own version of the special effects used in big budget movies.
So, I started the search for tools under Linux that would permit the green screen to be done.
As a short summary green screen/chroma key is where a a single colour is removed from an image.
Most often it is green as modern cameras are more sensitive to green and a bright green works best as it's less likely to be a colour in a natural scene. I remember when I was younger hearing it being done with blue screen as well.
After a bit of looking I found Imagemagick a jack of all trades image processing tool. It has a function to remove a single colour from an image and as importantly for my use includes a 'fuzzy' search for the colour which gave a bit of tolerance to the lighting.
In the code the most important line is:
os.system('/usr/bin/convert -limit thread 4 ' + folder +'imagecam.png -fuzz '+fuzzpercent+' -transparent "#'+rgbnum +' " ' + folder + 'imagecamt.png') | |
I know there is a lot going on in there. She short version is it takes imagescam.png, makes transparent the colour rgbnum with a tolerance of fuzzpercent and saves it as imagecamt.png
This forum thread was really useful in understanding how to use it.
Once I had my head around this it was then a matter of doing the rest of the code.
Take the picture
Remove the green
Layer a background and the image with the green removed.
For testing I used some of my kids PlayMobil and a piece of A0 green card.
All looks like it's working well even with the ability to change the background using the arrow keys
Now for the next level.
Scale it Up to Life Size !!!!
This of course means I needed a green screen background with standard that I bought from eBay. No idea when I'll use the black or white backgrounds that came with this kit, but I have them now.
And then Tweet the pictures.
For Tweeting I used tweepy and followed the excellent guide by Alex Eames @RasPiTV
Finally, wouldn't it be great to have a little remote control and not have to rely on a keyboard. Since the Raspberry Pi 3 has Bluetooth I thought this might be the ideal solution. No wires and no messing about. Since Pygame was already being used for the displaying of the images and I knew Pygame had joystick support built in this looked like the obvious choice.
Again, on eBay I found this small little Bluetooth gamepad and thought it would be perfect. Super small which means you can have it in your hand but not interfere with your final picture.
Iddy biddy, teeny weeny, black gamepad |
My usual style is before bring a new feature into a project I like to test it standalone to make sure it works. For the gamepad I created a small Python/Pygame program to test the gamepad
It paired with the Raspberry Pi first time and worked perfectly with the gamepad test program. So, 100% sure it will function with the chromaCam setup.
Now all the parts are in place. PiCamera to take the picture. Imagemagick to remove the background colour. Pygame to merge background, foreground picture with green removed and finally an overlay with the background images changed with the keyboard or the super small gamepad. (Oh yeah, the circular thing at the bottom is an analogue joystick)
The chromaCam set up got it's outing at the last Wimbledon Raspberry Jam. Here are some pictures.
On the day I used a camera tripod and a lot of tie wraps to hold the Pi Display and the Pi Camera onto the tripod. I would definitely recommend a more secure mounting method
Getting set up https://twitter.com/MrTomsWorld |
Who doesn't love a bit of Harry Potter - https://twitter.com/rdhayler |
Those Lions are dangerous |
That doesn't look like a sensible thing to do - https://twitter.com/HackHorsham |
Final notes.
If you did click the link above that explains Chroma Key/Green Screen then you'll have read that lighting is really important. The goal is to one specific colour. If your lighting is uneven then due to shadows or folds in the green screen the green will be different giving varying results.
Taking out too much - https://twitter.com/Codepope |
I'm sure the top of the Raspberry Pi logo isn't transparent |
A few bits of rogue green - https://twitter.com/gowolade |
If you want to make your own Green Screen Photo Booth then the code is available on GitHub
Final Thoughts:
Yes, this project worked and was great fun to build and see people using it.
For the project I limited the image size to 640x480 as even at that resolution it took about 2 seconds for the picture to be updated. This meant it took a little bit of patients to get the picture you wanted before pressing the button to Tweet the image.
I took the set up to Hack Horsham as well but I didn't bring my own lighting and the lighting in the room was perfect for a Jam but didn't give enough contrast for chromaCam to work. So, it didn't make the final cut and I stuck to Button Flash.
You can see the setup in the mannequin challenge video tweet.
@Raspberry_Pi please can we get a retweet? Mannequin video of today's #horshampijam pre-opening :) #rjam #horshamdc pic.twitter.com/g9v0jl7NeM— Hack Horsham (@HackHorsham) December 11, 2016
Again, this was a great learning experience for me and once the computing power for the Raspberry Pi enables it or access to the GPU is supported this will be amazing when the green screen transparency can be done in real time.
I expect as new devices come out I'll be revisiting this project to see if the performance is improved.
Any update on this with the new raspberry pi?
ReplyDeleteThanks for the reminder. I've a Pi4B 8GB with 64bit Raspberry Pi OS. Definitely interested to see if the faster Pi and more memory makes a difference.
DeleteI also have it setup with 32bit Raspberry Pi OS on an SSD where the speed increase for I/O could also be a massive benefit.
Are there some step by step instructions for this? I would love to recreate for a project I have in mind.
ReplyDeleteThere are more details on the GitHub page.
DeleteIt's not very complicated. Connect the camera.
Setup the folders are per the code
Add your background images.
Take picutres.
Be aware lighting is very important.
Thanks for taking the time to discuss that, I feel strongly about this and so really like getting to know more on this kind of field. Do you mind updating your blog post with additional insight? It should be really useful for all of us. Photo Booth Chicago
ReplyDelete