Motion tracking w bone rig?

chucky · Post by **chucky** » Thu Jun 11, 2009 3:52 pm

Not all good news guys, I tried this on a slightly grander scale, with two legs and arms , spine and the whole thing, I don't have the full version yet so posting examples is only ok for very short bursts (screen cap gif).

Although my first test EDIT: ( which I have not posted) was a simple and effective affair, I'm now not sure about how great it is on a full rig yet.
I definitely have some results that are really cool, especially as the foreshortening of the bones can result in some very natural body/head turn tricks.
Unfortunately the mocap starts to crap out when several targets and bones are in use.
The secondary target doesn't always 'grab' the bone's rotation.

Maybe I have done something wrong, like starting from other than the first frame or not laying out bones/targets in a correct hierarchy.

Advice to anyone trying this out, use good quality high rez video with clearly defined markers- hopefully multi coloured. The poorer the video the poorer the cap will be, and therefore the more adjustments you will have to make to the targets.

As for the bones..... I'm not sure why they flick around and change length and rotation in error, but if you get some of these symptoms just have a quick glance at the timeline graph and you will see some interesting artefacts. Sometimes the translation ' jitters' are easy to eliminate on the graph. Have a look.
Some the errors are (in the case of the bones) are hard to tweak as keys are laid down on every frame. Smoothing or some kind of pixel/angle tolerance could smooth that out.

A bitter-sweet experiment- mostly sweet...

heyvern · Post by **heyvern** » Thu Jun 11, 2009 6:34 pm

mkelley wrote:Now, we'll have to preserve the parent bone ID but doesn't the new scripting allow for that kind of data storage?

No, I don't think so but I've been so busy with other stuff I haven't had time to stick my brain inside the new scripting and rub it around to gain new knowledge...

... uh... that is how you guys learn stuff right? You rub your brain on things so the knowledge gets in faster? Or am I the only one who does that?

-vern

chucky · Post by **chucky** » Sun Jun 14, 2009 5:03 pm

OK so I haven't built the model properly but here's a mocap test, all the body movement is mocapped.
I have noticed the rotational values seem to double and cause that spinning effect much like the old bone lock glitch,this is most obvious when viewing keys in the graph.
If only I was asked to beta this feature.....sigh.

slowtiger · Post by **slowtiger** » Sun Jun 14, 2009 10:09 pm

That's ... amazing. Could you post one still of the motion source?

madrobot · Post by **madrobot** » Mon Jun 15, 2009 1:21 am

Wow.

That looks pretty good for a test Chuck.

(Plus I had no idea you had such bodacious moves.)

So the rotational issue you're talking about, did you go in and correct it?
Or is it there in this file, and doesn't show in the render?

The dynamics on the hair really works haha.

re: beta testing, I agree. I think it's great that Vern was on it, but having the whole posse of yourself, Wes, mkelley, genete, slowtiger and probably one or two others who have slipped my mind would have been in the best interests of ASpro users for sure. Imagine what that might have looked like!

Still, we'll see how we go...

Patmals · Post by **Patmals** » Mon Jun 15, 2009 2:22 am

That looks really cool, Chucky!

Please show us how you acheived this;)

madrobot · Post by **madrobot** » Mon Jun 15, 2009 2:52 am

I'd be interested to know what measures you took to record the source video. Dark clothes against white wall? What light? Tracking objects?

chucky · Post by **chucky** » Mon Jun 15, 2009 4:07 am

the source video could have been better,
I captured through vegas pro using my wbcam for want of something better.
I used masking tape in bands around main joints and targets, I just wore what I had on which was on the dark side(wheez), and the lighting was totally insufficiant, it was already evening.
I did ajust the contrast in vegas and added a gradient filter which looked a bit like predator vision just to highl;ight the markers.

If I did this whole process properly with good lighting and dark BG , I'm thinking multicoloured reflective markers will make the mocap targeting quicker and more reliable.
Of course when the elbow for example goes behind the body, I had to cut a huge hole through my stomache to see through to the back.

That was messy and I've got to say, really smarted.

I will post some more details regarding timeline issues etc tonight when I get home.

Mikdog · Post by **Mikdog** » Mon Jun 15, 2009 10:15 am

Dude. This looks awesome. I'm gonna watch his thread closely. Very cool, guy

A simple tut on how you set the bones and layers up would be super duper.

chucky · Post by **chucky** » Mon Jun 15, 2009 12:50 pm

...............

.........................

Yes , yes very silly, you can stop laughing now, if you are can.
And no there is no 'krumping' take, that I will admit to.
Ideally as I said before good lighting, dark/mid tone backdrop and various coloured reflective targets would make targeting faster and cleaner.
Also target placement especially around the feet could have been better.
I have been tracking one target at a time ATM so as not to get in a tangle.
A cool feature is that you can use the same source many times and track indidual parts as the bones will track targets on any number of video tracks.
Make sure the bones you want to control are all parent bones as their children won't be able to track.

Give it a shot, we could work together to find the best system for this.

This is the rotational jump that causes ' bone flipping', although an undesirable outcome, this can be fixed by dragging the keys in question to be in line with its corresponding keys. I'm not sure what causes this, hopefully we can find a solution that will reduce the need for this tweeking, it is one of the things does interupt the process somewhat.

I think a script might be able to smooth all these keyframes, but it would have to be descriminating.

There are a couple of other small issues which I won't delve into just yet, these are still early days and I keep finding ways to avoid them.
More soon.

realsnake · Post by **realsnake** » Mon Jun 15, 2009 10:26 pm

whoa super cool, i had no idea what exactly is tracking point tool and its uses, haven't used any video editor like after effect before.

EDIT: This is another form of motion capture in 2D, instead of rotoscoping and tracing over videos

dreeko13 · Post by **dreeko13** » Wed Jun 17, 2009 1:52 pm

i think what is needed here is a mocap bones tutorial for dummies cause this has so much potential

with a mocap from a close up video of a mouth talking (with lips marked at the corners and middle upper and lower) then , we would have instant lipsyncing!!

windstormer · Post by **windstormer** » Thu Jun 18, 2009 12:03 am

Perfect! I'll start experimenting now. These is really just one of the features that makes this software rock!

chucky · Post by **chucky** » Sat Jun 20, 2009 5:28 pm

I think this post has long outgrown the 'how do I' but I'll continue anyway.

Here's a face motion capture test, interesting experiment but a bit unweildy.

Huge Orange Head

It all works well but it got a bit slow as the mouth movement was too subtle.
I had to use controlled bones with amped values to exaggerate the movements.

I think mocap is going to be generall better for gross body movements.
The facials a better just with the pose sliders- much quicker and more maleable.
It may , however be good for more realistic characters, I may have a look into that.

realsnake · Post by **realsnake** » Sat Jun 20, 2009 8:26 pm

whoa super duper cool