Rotoscoping in Moho pro 13.5.2

[Mattyj]

Is it possible and/or would it be worth it to attempt a Rotoscoping effect in the current build of Moho? Just interested.

[slowtiger]

Depends on the look you want to achieve. Usually rotoscoping is done in bitmap programs like TVPaint because in essence it's FBF, and there's lots of elements showing up in one frame and gone in another - not the field where Moho shines. OTOH rotoscoping can be seen as a way of motion capture, and with a rigged character in Moho you can quite easily follow a viodeo reference layer.

[Mattyj]

Got it cool cheers slowtiger

[Greenlaw]

Mostly just opinion below based on personal experience but FWIW...

I like to use rotoscoping in 3D animation when I need realistic motions and I don't have mocap available. As a matter of fact, we used to jokingly called this technique 'roto-capture'. Back then (late 90's to early 2000's), real motion capture was super expensive, and for us this was the 'affordable' form of mocap.

To do 'roto-capture', it's just a matter of importing the reference footage on an image plane or 2D card, and then animating the rig on top of it. For 3D, you ideally want the footage shot from multiple angles, and for the images to be precisely aligned and playing in sync. Just like with regular keyframe animation, when moving through the footage, you want to find and set the 'key poses' first, and then go back and pose the in-betweens. This ways, the result will be a smoother animation without a lot a 'chatter'.

Also, you want to use JPEG image sequences for the footage because this format is 100% frame accurate and it streams well even across a network. You can use a low JPEG quality setting (30% - 60%) to improve performance. Seriously, it's absolutely unnecessary to use higher quality setting for 'roto-capture'.

These days, I don't bother with animating on top of the footage; I just place the single camera footage in the corner, scaled to thumbnail size, so I can use it as a visual reference. This leaves me with room for interpretation, which tends to make the animation look aesthetically nicer. But if I'm creating the animation for a 'live action' visual effect, like for a digital stunt double, I'm probably going to follow the original motion in the footage pretty closely. As they say, you can't get more real than the real thing.

It's possible to use this technique for 2D puppet animations but personally speaking, I don't use 'roto-capture' in Moho because the characters I animate in Moho tend to be very cartoony, and realistic motions would look weird on them. Plus, it's easy and fast enough to animate cartoon styles in Moho, and adding roto to the process would just slow me down. But then again, for realistically designed and proportioned 2D characters, I might consider importing live footage to use as sight reference rather than for tracing, as described in the previous paragraph.

Now, if you want to use motion capture, for animation that's a whole other thing.

First, my experience with mocap for 3D animation.

I've been involved with using professional motion capture studio for video game cinematics and TV commercials, and I personally use a 'home-brew' mocap system (using synchronized depth sensor devices and iPi Mocap Studio) for my personal shorts like Happy Box and the Brudders music video, and for vfx work in some low-budget motion pictures. (My current setup uses two Azure Kinect Sensor, three PS Move Controllers (for head and hand rotations,) and iPi Mocap Studio.)

For me, motion capture is a fun but complex process, and probably not for the feint of heart. You not only need to understand 3D rigging and animation, you also need to understand how to retarget data from the mocap rig to your character and edit it...the rules are very different from animating a character by hand, and you're relying a lot more on animation layers and using constraints.

It's actually a lot of works, but I find motion capture tech is a great tool when you have a lot of characters animate and a tiny crew of artists. For example, when I worked on Call Of Duty and Star Wars Kinect cinematics, we had hundreds of characters to animate and a crew of maybe 10 artists. We still wound up doing a lot of hand keyframing anyway but without mocap, these productions would have been impossible to complete on time.

Or, when using mocap in my personal animation and b-movie work, there is only one artist...me. I typically use my 'home brew' mocap system for fun experimental projects and occasionally for digital stunt work in live action movies and commercials. It's hard enough compositing live/cg and creating VFX on tight deadlines with limited funds, and using a mocap system literally shaves off weeks off my schedule that I might otherwise spend animating the characters by hand. It's also very convenient to be able to step into our living room and record new motions as I needed them!

Now, here's my experience with mocap for 2D animation...

I still use my budget mocap system for personal projects but not for 2D animation. That said, budget motion capture systems do exist for 2D animation.

The most common and probably cheapest workflow is Adobe Character Animator, which uses a webcam to record facial expressions and simple puppeteering. I've dabbled with this software but don't have enough experience to fully comment on it. There are probably a lot of videos about it on YouTube.

Currently, I use Reallusion's Cartoon Animator Pipeline, which has more robust capture hardware options available. The idea behind this program is you create your character art using a layered PSD template, in a paint program like Photoshop; next you import that artwork and rig it using one of the standard mocap rigs in the program; and finally, you record your motions and face expressions using a web cam, or if you want more realism you can use an iPhone 11 or higher model (using the built-in face tracker,) to capture face and limited body motions, and a Leap Motion device for hand/finger capture. You can also use some body mocap systems or existing mocap data to animate the rig.

I'm currently experimenting with this system (using iPhone and Leap Motion) to capture motions for a 'host' avatar in my tutorial videos. I think the results are okay when you're aware of the system's limitations and consciously play into it. (The same is true for the 3D mocap system I use in my short films and low-budget film work.)

TBH, while Character Animator is fun to play with and I think it will be useful for certain types of projects, it's also a hassle to use if you want to produce complex or 'quality' animation work. For example, to do any custom 'hand keyframed' animations is really awkward compared to Moho, and CA doesn't have nearly the advanced rigging capabilities and options that you get in Moho. Back when it seemed Moho was going away, I asked the developers to improve on the program's keyframing ability but they really seem more interested in animating with performance capture and using existing mocap data. That's fine I guess, but a bit frustrating when you just want to animate something by hand. But if you're willing to accept the limitations and design your productions around them, I think CA should work well.

Moho doesn't officially support motion capture for characters but it does have a motion tracking system than can be adapted for it. Chucky came up with such a workflow and did some impressive experiments with it. A search in these forums will turn up the examples.

For hand keyframing puppet animation, Moho still wins at every level, even against Toon Boom Harmony or After Effects with DuIK. These programs are better for what they specialize in, FBF animation for Harmony, and vfx/motion design work in Ae, but for rigging and animating all-purpose 2D puppets, Moho is way more capable, faster, and easier to animate with...even without relying on motion capture or auto lip sync assistance.

Hope this info is helpful or at least interesting on some level.

[Greenlaw]

Oh, I almost forgot...my child Sienna showed me an interesting 2D puppet mocap system designed for live streaming or vTuber videos. Two programs I'm aware of are Live2D and vTuber Creator. Here's an interesting demo video for Live2D.

Live2D Demo

I think Sienna had been playing around with one of these programs...I'll ask them about it.

I don't think this kind of performance capture animation is particularly useful in creating animations for traditional cinematic-style storytelling, but it's pretty cool and obviously related to how puppet animation programs like Adobe Character Animator and Reallusion Cartoon Animator Pipeline work.

IMO, this is fine for live streaming, but for cinematic storytelling I still think Moho is the better choice. For example, I really can't see creating any of the 2D animations shown on my demo reels using one of these other programs.

[chucky]

Oh, I almost forgot...my child Sienna showed me an interesting 2D puppet mocap system designed for live streaming or vTuber videos. Two programs I'm aware of are Live2D and vTuber Creator. Here's an interesting demo video for Live2D.

Live2D Demo

I think Sienna had been playing around with one of these programs...I'll ask them about it.

I don't think this kind of performance capture animation is particularly useful in creating animations for traditional cinematic-style storytelling, but it's pretty cool and obviously related to how puppet animation programs like Adobe Character Animator and Reallusion Cartoon Animator Pipeline work.

IMO, this is fine for live streaming, but for cinematic storytelling I still think Moho is the better choice. For example, I really can't see creating any of the 2D animations shown on my demo reels using one of these other programs.

But Dennis, the user might want a particular style of image and animation than ones that would use those techniques.
There are some interesting and stylish uses of roto, especially in music videos and arthouse stuff.

You could do a mix of FBF and use the redraw vector for some interesting tweening between frames, brushes and masks could do some interesting stuff too with vector noises and the like.
Hmmm the possibilities.

[chucky]

Oh, I almost forgot...my child Sienna showed me an interesting 2D puppet mocap system designed for live streaming or vTuber videos. Two programs I'm aware of are Live2D and vTuber Creator. Here's an interesting demo video for Live2D.

Live2D Demo

I think Sienna had been playing around with one of these programs...I'll ask them about it.

I don't think this kind of performance capture animation is particularly useful in creating animations for traditional cinematic-style storytelling, but it's pretty cool and obviously related to how puppet animation programs like Adobe Character Animator and Reallusion Cartoon Animator Pipeline work.

IMO, this is fine for live streaming, but for cinematic storytelling I still think Moho is the better choice. For example, I really can't see creating any of the 2D animations shown on my demo reels using one of these other programs.

But Dennis, the user might want a particular style of image and animation than ones that would use those techniques.
There are some interesting and stylish uses of roto, especially in music videos and arthouse stuff.

You could do a mix of FBF and use the redraw vector for some interesting tweening between frames, brushes and masks could do some interesting stuff too with vector noises and the like.
Hmmm the possibilities.

[chucky]

Oh, I almost forgot...my child Sienna showed me an interesting 2D puppet mocap system designed for live streaming or vTuber videos. Two programs I'm aware of are Live2D and vTuber Creator. Here's an interesting demo video for Live2D.

Live2D Demo

I think Sienna had been playing around with one of these programs...I'll ask them about it.

I don't think this kind of performance capture animation is particularly useful in creating animations for traditional cinematic-style storytelling, but it's pretty cool and obviously related to how puppet animation programs like Adobe Character Animator and Reallusion Cartoon Animator Pipeline work.

IMO, this is fine for live streaming, but for cinematic storytelling I still think Moho is the better choice. For example, I really can't see creating any of the 2D animations shown on my demo reels using one of these other programs.

But Dennis, the user might want a particular style of image and animation , other than ones that would use those techniques and applications.
There are some interesting and stylish uses of roto around , especially in music videos and arthouse stuff.

You could do a mix of FBF and use the redraw vector for some interesting tweening between frames, brushes and masks could do some interesting stuff too with vector noises and the like.
Hmmm the possibilities.

[Greenlaw]

Yeah, it's more of a personal preference I guess, and I'm sure I can be convinced otherwise with good production-quality examples. And these do exist.

One of my coworkers worked on a pilot series where they mainly used Adobe Character Animator with After Effects. I think the series might gotten tied up some legal issue and was never released, but from bits I'd seen, it was funny and looked pretty good. And this was years ago when ACA was still in beta.