Wiimote Gesture Recognition

Up until this post, I assume that you should have played with your wiimote for some days.
You probably have tried to program something with the wiimote library that I've introduced from the previous post.
Now you might come with a question that, how to actually make sense from the data of the accelerometer on the wiimote.
So there are people doing gesture recognition with the wiimote.

I've just realized that to fully utilize the accelerometer that I actually need a gesture recognition software when I was reading the wiimote specifications at Wiili.org.
So the first library that I've discover is the Wiigee which is a java based library.

wiigee is an open-source gesture recognition library for accelerometer-based gestures specifically developed for the Nintendo® Wii™ remote controller. It is implemented in Java™ and, thus, is platform-independent. Using a third-party Bluetooth®-library wiigee allows you to define and recognize your own, freely trained gestures.

The wiigee library

• allows you to define (train) your own arbitrary gestures,
• recognizes these gestures with high accuracy,
• offers an event-driven architecture with which you will be able to
• integrate the gesture-input as easy as common mouse-input.

Here come some video demo for wiigee.








Even though I've downloaded the library but I haven't got it to work on my pc yet.
As the library require another bluetooth library for java and I've always got an exception about the bluetooth connection stuff.
If any of you manage to get it work on your pc, please don't hesistate to give me an update.


Here there is a library for C/C++ which is called Accelerometer Gesture Recognizer.
I still haven't got it work with my pc either as I still cannot manage to compile it due to unknown reason.

However, there are few alternative C/C++ libraries to it such as Accelerometer-based Gestures for Openmoko Neo FreeRunner which also works for wiimote.
This project was done under the Google Summer of Code 2008 program.



Even though I've never tried to build this library as it is written in C code though but it seems work pretty good from the video demo above.

There is another C/C++ library which is program by a Taiwanese which is WiimoteGR (I found his blog which lead me to this page anyway).
On the wiki page which is linked above, the author even describes how the library work in diagram and some pseudocode.
It is pretty good reference if you want to program a similar library.
I guess that I will use this library for my project as it works fine when I tried to train it with some gestures and have it to recognize the gesture.

Finally, here there is a C# library for those who love managed code which is Wiigle.

Wii-based Gesture Learning Environment
WiiGLE provides support for recognizing gestures based on the Wiimotes accelerometers. The current version features some basic feature calculations on the raw signal and a simple NaiveBayes classifier. To this end, it makes use of the Managed Library for Nintendo's Wiimote by Brian Peek and the Weka Data Mining Software by the University of Waikato. It provides interfaces to integrate your own feature calculations and your own classifiers.

WiiGLE provides and environment to record gestures, calculate features, train classifiers, and use the Wiimote for online gesture recognition based on accelerometer data. It features a Programming API that allows integrating your own features and classifiers.

http://mm-werkstatt.informatik.uni-augsburg.de/documents/WiiGLE/doku.php

There are some good documentations on it and even a video tutorial.
It seems pretty even though I haven't tested it out yet.


So for the few libraried above, most of them are released under LGPL license which allows you to do anything with it either open source or commercial.
This is a good sign for those who want to learn more on it.
You can just download the source code and see what the authors have done.


So now you have the gesture recognition libraries here, let's get it Rock 'n Roll!!

How to make use of the Wiimote

So now you have your wiimote connected to your PC, isn't it?
You probably tried to press the buttons on the wiimote or even tried to shake it, but nothing happens in the end and the 4 LEDs just keep blinking.
Don't be frustrated, it is because you do not have any programs that read the data from your wiimote and make use of it.
So here I introduce to you few of the wiimote libraries that I've tested anyway.

First, here comes the managed library which is WiimoteLib.
This library started out as an article at the Coding4Fun site and eventually many people use this to develop their projects with wiimote.
For a familiar example, it is also used by Johnny Chung Lee's projects which were posted in the previous post.
Here's a screensot to let you see how it is looked like.



Next, here comes a C library called Wiiuse.
It is one of the very first library that enable you to gain access to the wiimote with your PC, I guess.
For those who like to program in C language, you can go for it as its support for the wiimote is pretty good and it can support most of the bluetooth stack theoritically.



In case that you would like to program in Java, there is a Java wrapper for the Wiiuse above called WiiuseJ.
It is supposed to be able to do whatever the Wiiuse can do though.


Lastly, it is a great news for those who love C++, there is an excellent library for C++ which is WiiYourSelf!.
It is a fully featured library which supports everything from the wiimote even the speaker and vibrator.
This is the library that most recommeded by me as it is really easy to use though.




Here I've introduced several libraries tested by me.
In case you want to know more other libraries available, feel free to visit the URL below.
http://www.wiili.org/index.php/Wiimote_driver

How to connect the Wiimote to PC

If you were reading the previous post, you probably want to know how to connect the wiimote to your PC.
Here there is a simple trick to do that.
First, please make sure that you have the ingredients (cooking?) below on your hand.

• Wiimote - I assume that you should have it on your hand now
• AA batterry x 2 - It is required for your wiimote to work anyway
• Bluetooth dongle/device - Here there is a link to the Wiili.org that list all the compatible bluetooth devices (http://www.wiili.org/index.php/Compatible_Bluetooth_Devices)

So, I believe you should have them on your hand to start with.

1. Install your newly bought (probably) bluetooth dongle with your PC. Make sure that you have installed the driver properly and it has a working bluetooth stack. For my case, my laptop has an integrated bluetooth device and I installed Toshiba Bluetooth Stack on it. In case you are using Windows Vista, the Windows Generic Bluetooth Device also works actually.
2. Now your bluetooth is working on your PC, put 2 AA batteries into your wiimote and press some buttons to make sure that it is in working condition.
3. Up until this step, you are probably very excited and trying to scan for bluetooth devices on your PC after you press the 'on' button on the wiimote, don't you? And you are frustrated with it that the wiimote has a very short time out that your PC is not detecting it as fast.
4. Here there is the trick comes in, press the 1 and 2 buttons on the wiimote simultaneosly and have your PC to scan for bluetooth device immediately. With this trick, the time out for the wiimote becomes longer and your PC should be able to detect it and establish the connection. In case that your bluetooth stack ask you for the type of the device being connected, you should try to seek for the "Human Interface Device" from the list the options it gives you.
5. Daadaaaaang!! Your wiimote is now connected with your PC if you are able to proceed through these few steps without any problem. A prove of your wiimote is connected is that the 4 LEDs on your wiimote should keep blinking now.

So, is it really easy to get your wiimote connected?
I hope you feel so.
This ends this post.
Stay tune for the next post: How to make use of your wiimote?


Edited: Added a video from youtube as a demostration.

Final Year Project

It had been 2 years that I abandoned this blog.
I decided to revive this blog and use it to post some of the technical stuff that I have a touch with it.
So let's start a little bit with it.

In case you do not know that, I'm under going my final year project (here onward refers to fyp) in the university.
My fyp title is "Developing Interactive Application with Nitendo Wii Remote" which is supervised by John See.

Above is the picture taken on the fyp online system which is listing some detais of the project.

So now we go to the main point.
Why wiimote?
What's so special about it?
I believe you might not see very cleary on the descriptions written on the picture above, so I have to go over it by myself.

A snapshot of Nitendo Wiimote.

The Wii Remote, sometimes unofficially nicknamed "Wiimote", is the primary controller for Nintendo's Wii console. A main feature of the Wii Remote is its motion sensing capability, which allows the user to interact with and manipulate items on screen via movement and pointing through the use of accelerometer and optical sensor technology. Another feature is its expandability through the use of attachments.

quote: http://en.wikipedia.org/wiki/Wii_Remote


Basically, wiimote is an innovative controller developed by Nitendo for their latest console which is called Nitendo Wii.
Wiimote posseses capabilities as stated above that others do not.
Due to the fact that the wiimote is connected to the Wii console with bluetooth, people have tried to connect it to the PC and they succeeded.

They managed to decode the packet sent by the wiimote and retrieve the information within it.
It made using the wiimote with PC as a new human interface device becomes a new possibility.
There are few interesting projects that utilized the capabilities of the wiimote.
Few examples are given here.

Tracking Your Fingers with the Wiimote

Low-Cost Multi-point Interactive Whiteboards Using the Wiimote

Head Tracking for Desktop VR Displays using the Wii Remote

quote: http://www.cs.cmu.edu/~johnny/projects/wii/


Above are few very interesting projects done by Johnny Chung Lee.
There are a lot more of interesting projects that are available on the Internet and most of them have a video uploaded to youtube to demostrate it.


So back to the discussion of my project, the main point of my project is to mase use of the wiimote to create an interactive application.
Here the application could be anything else but I've proposed a dodge ball-like game for it.
Due to the nature of this project, it would be mainly based on the implementation rather than research.
And my main focus on this project is interactivity.
So to start everything, I need to have a wiimote on my hand.
Edwin (tutor that is the co-supervisor of this project) lends me one of his wiimote for us to start with it.

Wiimote that I got from Edwin last week and 4 AA batteries that I bought from the shop at FIT.

The above picture shows the equipments that I need to start with.
The next step is about to connect the wiimote to my pc and run some demos with it.
So stay tune for the next post.