5
$\begingroup$

I'm fairly new to machine learning so please pardon me if I mess up terms. I'm working on a wearables project that involves hand gesture detection, looking for any pointers on potential algorithms that would be applicable.

Basically there will be sensors that pick up tendon movement at the wrist to provide input, so for simplicity I'm assuming that would mean 5 features (one for each finger), each with values ranging from say 0-100.

There are also 5 classes that the gesture could be classified as. The user would do each type of gesture several times to provide training data, and then for each gesture class I'd need an algorithm to determine the best model for it based on that training data.

After initial training, whenever the user does a gesture it would input the sensor readings into each model to see which one is the closest match. Additionally, since this is a wearable anything that optimizes for low memory and low processing power would be best.

If anyone has any input on either the type of models or the algorithm with which to generate them that I should look into I'd appreciate it!

Edit: Here is some sample data from 4 sensors during 3 different gestures. These are basically just pressure readings from sensors lying along tendons on the wrist.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

It might be really useful to provide a bit more details regarding the actual data, produced by your sensor. Can you provide a simple example for any simple gesture?

Anyway, let's try to address the problem in the following way:

  • sensor can be represented as a function f(t)
  • range of the function will be {1,2,3,..100}

So we can draw something like the following (can we?):

enter image description here

You can extract various high-level features out of that data, but I would suggest to go with inflection points as the most straightforward option:

enter image description here

Those points, basically, define a pattern. I'll use a figure from the different answer to illustrate the idea. For example, the naive pattern below is used to identify a circle.

enter image description here

In your case situation is a bit simpler, because sensor output is the only data you can work with. Naturally, you can combine multiple sensors in order to define a bit more complex interaction scenarios:

enter image description here

Also you can take into account f'(t) in order to distinguish between slow and fast movements, etc.

Update

Thanks for the data. It does look like these three gestures looks pretty much the same from the data perspective (see visualization below).

enter image description here

It really looks like several features are required to identify the gesture.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Awesome, thanks for the response. I edited my question to add sample data $\endgroup$ Commented Nov 19, 2015 at 3:04
  • $\begingroup$ @Luke, did you manage to solve the problem? See update, I visualized the data (sorry for the delay). $\endgroup$ Commented Jan 24, 2016 at 19:37
0
$\begingroup$

The types of methods that would work depend on the nature of the data being captured in relation to the output classification. If the gestures are linearly separable in the 5-dimensional input space, a simple logistic regression is cheap and fast.

Alternatively a weak but very cheap method would be to find the mean input vector for each of the gestures, and simply decide which gesture vector the input is closest to at any point. The downside there you might need another class for "not gesturing."

It's also possible to gather a lot of test data from many users to pre-train a model, and long as readings don't vary too drastically between users,you wouldn't need to train on a by-user basis (on the wearable itself). In most cases prediction is efficient, so you wouldn't be too limited by computational power.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the response, edited my question to include data $\endgroup$ Commented Nov 19, 2015 at 3:05

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.