Mask Bug - work area and render engine Different.

Discuss Moho bugs (or suspected bugs) with other users. To report bugs to Smith Micro, please visit support.smithmicro.com

Moderators: Víctor Paredes, Belgarath, slowtiger

Post Reply
User avatar
Rhoel
Posts: 844
Joined: Fri Feb 25, 2005 8:09 am
Location: Phnom Penh, Cambodia
Contact:

Mask Bug - work area and render engine Different.

Post by Rhoel »

I reported this before but forgot to post the demo scene.

The scene is the bouncing ball test. The shadow is masked to the ground so when the ball bounces over the edge, the shadow is half/half.

In the moho work area window everything looks fine. Render the beggar and the ball is masked too, even though its layer settings are clearly marked as ignore.

This is one area of Moho where a little more work is needed.
My personal thoughts are:
  • The mask must be external to the layer. This would make the renderers task of mask or not mask much more practical.
  • The mask must be easily reverable, so it can be applied as a foreground or background mask (reveal/hide).
  • Which layer is used to mask should be selected from a dropdown list of scene layers. This way, one mask can be used to matte many layers.
  • Having more than one matte on an ojbect would be good if it is technically possible. There are many occassions in the past when i have needed multiple mattes.
  • An option if the fil and line or just fill is to be used as the matte: This is because there are occassions when you want to see an outline - an eye ball has an outline - you may or may not want to see the black outilne during the blink.
Some of this might not immediately appear obvious as to the requested reasons - But recently, I discovered when doing High Definition tests that I could see matting errors previously hidden in Standard TV: Specifically, I make my mouth shapes using masks and switch layers. I could see the matteed in tongue had no outline but the mouth shape did. You cannot see that on SD.

The test scene is here: Mask Bug demo
User avatar
Lost Marble
Site Admin
Posts: 2354
Joined: Tue Aug 03, 2004 6:02 pm
Location: Scotts Valley, California, USA
Contact:

Post by Lost Marble »

Yes, this is a problem that the work area can look different from the rendered result. In this case, the rendered result is doing the correct thing, and the work area is wrong.

In the sample file, the Ball group should be masked as a whole against the BG layer. If you think of a group layer as a black box, it should all be treated as one object. The rendered view does this - the whole Ball group is masked as one.

What you're trying to do is have a mask outside of a group affect the layers of the group in different ways, and this isn't possible. A group should always be considered as a single object - that's the way anything outside of that group (including masks) thinks of it.

What you see in the editing view is the actual bug here. Although it looks like what you want, it isn't displaying correctly.
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

Yes!

I found that if I "go with my instincts" I get the correct results with masking.

When doing any masking always do a quick render as you "set it up".

I have been able to do more with masking since learning to "ignore" what I see on the screen till I render.

Don't be afraid of using a bunch of group layers for masking. You can do amazing stuff.

I found the simple act of putting masked group layers inside another group layer and changing the masking for the whole thing... gets me some really fun masking tricks I thought weren't possible.

Experimentation is the key here. Setting up a very very very simple masking test file... and trying all kinds of possbile combinations... goes a long way to figuring it out.

I use that test file when I have a really complicated trick I need to do. Once I get it working there it is easy to transfer that to the more complicated file.

-Vern
User avatar
Lost Marble
Site Admin
Posts: 2354
Joined: Tue Aug 03, 2004 6:02 pm
Location: Scotts Valley, California, USA
Contact:

Post by Lost Marble »

heyvern wrote:I have been able to do more with masking since learning to "ignore" what I see on the screen till I render.
Simple masking (with a bunch of layers in one group) should display fine in the working view. It's just when you get into multiple layers of groups with different masking options that the editing view becomes unreliable. Complex masking like that will get easier when the editing window is able to preview it correctly.

The key to complex masking (and a lot of other things) is to always think of a group as a single object.
User avatar
Rhoel
Posts: 844
Joined: Fri Feb 25, 2005 8:09 am
Location: Phnom Penh, Cambodia
Contact:

Post by Rhoel »

Lost Marble wrote:What you're trying to do is have a mask outside of a group affect the layers of the group in different ways, and this isn't possible.
Thanks Mike.

I understand the mask paramaters and the operaltional limits ... what is desiable in 5.x or 6.0 is the ability to use an external mask on a group - there are very big advantages in doing this. The way the composite is set up is the "proper"way to do this - that is, all the moving elements of a character (in this case the ball and its matched shadow) have only one set of size, x,y,z movement keys. Under the present way of working, the mask would have to be inside the moving opject, which means it will move and therefore not match the stationary background. This is the problem which needs adressing.

In some series films using stock backgrounds, I might expect as much as 15-20% of scenes to use this type of stationary matting against a moving character - it replaces the old match line common to the old ink and paint days. The usual scenario is a character walking behind a table painted on a background. At present, the only way of solving this is to make a cut out in Photoshop then import as a new layer: This is unsatifactory since any changes/camera moves/depth of field settings have to be changed.

The suggestion in the original post is a probably solution - having a selectable matte for the group layer and electing which sub-layers to accept or reject the matte. It would make the masking/matting use much simpler.

Just tryoing to help with the complex task of software design, providing insight into how a feature is used in practice.
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

I am trying to get my head around this idea...

You have an image as a background.

This image has a table in it. The character must walk behind the table and thus be masked out of the table. So you could use an alpha to mask out the character from the table.
At present, the only way of solving this is to make a cut out in Photoshop then import as a new layer: This is unsatifactory since any changes/camera moves/depth of field settings have to be changed.
This is the part I don't understand. Why precisely can't a second layer with the chair by itself be used? What does camera moves or depth of field have to do with it? Can't you set layers to ignore those things?

If you want the table to be a mask for the character and not move... then... just put it outside any groups that move. You might have to put the whole thing in a "outer" group and have masking set there...

Does this have to do with turning masking on and off? Having to use one or two "extra" layers to achieve the effect?

Maybe I missed the point somewhere. It would be nice if there were an animated example of this situation.

Seriously, I keep hearing about this scenario but I have never fully understood the problem.

I have masks for objects that are stationary and characters that move around them all the time... of course to be in front... I need to have a duplicate layer... that is fairly... easy... from my point of view... it would be nice to NOT have this extra layer of course.

-Vern
User avatar
Rhoel
Posts: 844
Joined: Fri Feb 25, 2005 8:09 am
Location: Phnom Penh, Cambodia
Contact:

Post by Rhoel »

Hi Vern,

NB: My mistake on Depth of Field - you wouldn't be doing DoF mods with a matched background, though you might have other effects to worry about.

Okay, this answer is going to get long - you might want to make some coffee first.
It would be nice if there were an animated example of this situation.

You do, you have the bouncing ball scene posted above.

It comes from an actual working example I used to use for training compositors with Animto 1.8 and 3.x. The exercise tests both the compositor and software, and introduces newbies to one critical element - the director!

Directors are one rogue part of the animation process, those un-namable people who delight in messing everything up because they change thier minds, frequently and repeatedly.

As a result, in the real commercial world, you have to make sure everthing is kept together, KISS. Character A in one folder/layer, Character B in another. That way, you can reposition, resize, change opacities or whatever the little whim these fickle monsters have called for. Backgrounds and anything attached to it are in a single folder. Overlays are treated in the same way - you can be sure the director will request a change to the out of focus settings (and you will have to change it back again when he sees it on screen).

In the bouncing ball test, the first stage is to make the ball bounce on the spot, squashing on impact, and the shadow looking realisic (its size and opacity have to be changed too) and in sync with the ball. Then you tell them the director has changed his mind and wants the ball to bounce through shot. The compositors have to change the east-west positions and timings to make it work as per new storyboard. Then you tell them the director from hell wants the ball to bounce front-left to rear-right. That means they have to get into scaling (the secret is to make the scale centre at the point of the shadow).

But he's not satisfied: Our bloody-minded leader (who couldn't direct traffic, let alone films), thinks its going to be great and funny if the ball falls off the table top and plummets to digital oblivion: This is the stage the bounclinng ball scene above is now at. It means matting the shadow without affecting the ball. In Animo, matting the shadow is easy - the ball is more tricky - it has to have its matte reversed/turned on just before it diappears behind the table surface.

Unfortunately, Moho failed the test when I tried matting it ... you cannot use the same matte on two objects. Neither can you use the background layer as matte on the shadow because its outside the layer: Put the matte inside the layer and it wont work because the layer is moving. You could match-move it if it was a linear move but put the ball onto a curved track and you will never match-move the matte with the background.

Okay, so you could dupe the background and import it over the top. As the director of this test, I'll mess you up - I want the angle of the background to be changed. Now you have to remake/change two layers. If the matte was external to the layer, you could make the change safe in the knowledge it will self-matte and no other changes are needed.

Okay - lets look at the final level of this animation assignment. The director wants the ball to bounce off a new vertical wall, then drop off the table. As Moho is now, even using a cloned layer to do this is very problematic ... I have never tried it and to be honest, I can't see any point in wasting time "fudging it". It requires a different approach.

Being fair, this is a very carefully designed assignment whose aim is to demonstrate all the principal composite tools. When you first start, it looks childishly simple. Then the tutor then introduces the director changes and the smiles are wiped off thier faces: Unless trained, people normally build composites without anticipating modifying it later. If you set every scene up so it can be changed later, your collegues will appreciate it. Usually, by the end of a trianing session, the students fully understand why you need to keep everything organised, elements kept together when ever possible, and always expecting the fickle-minded director to request changes. It's just good working practice.

So what about mattes in practice, for real in the field ... what is the director going to throw at you, and always at the last minute?.

For one, he will tell you its digital and therefore its easy to change. So change it he will. Colours, effects, camera moves, camera shakes get put in, then taken out again, etc. You do not want to rebuild something every time. So it can be better to add an empty layer with camerashake or modified move so you can revert to the old move if he changes his mind and reverts back - believe me they do and have, and want that exact move he liked before to changed it.

In a 10 minute short called Drums of Noto Hanto (for S4C, UK), the Animo team had to complex-matte every scene - every fill and line on every background and character was a matte replacement. Here is an example:

Image

Everything you see in this image is texture matted in, everything - every fill, colour and line: Not one single line of the original animators drawing remains. If a scene had just one background and one character, the matte count would be around 60 mattes. On the most complicated scene (Sc 12 if anyone is interested), I put in over 300 mattes, in a scene with three separate levels (levels not layers - BG, midde ground and overlay groups). The scene is a vary speed pan tracking a character from A to C pegs (16 inches). When you get to this level of complexity, you quickly understand why everything HAS to be grouped in one folder/layer, with just one control point/timeline. In addition to the matting, most exterieor scenes had 10+ multi-plained airbrushed "mist" layers, with the characters standing in the middle of the ground-hugging mist (aka music video dry ice fog).

Not all films are that complex.

But in the commercial world, the art director will ask for something which looks different. It means the composite has to be KISS. Self-matting layers is just part of that. Cloning isn't. Photoshop-ing a background is practical but matteing the object is better - its faster and the matte can be applied in multiple ways, ways in which a alpha channel cut-out can't. Softedge mattes is one (flame glows intrude on an object in front - a cutout will give a hard edge).

Moho has some great tools in it and I love the program. It can give you a variety of differnt looks and feel to a project: The 3D aspects and pan-arounds are really good - it made my producer sit up and slap people on the back, "a job well done".

But there are elements of Moho which, if being honest, are clunky. This test demonstrated three. The ease-in/ease-out maths didn't work correctly. The work-area dispaly failed to provide WYSIWYG. And the test exceeded the masking capabilities.

I hope people will not see this observation as being overly critical. It's in fact about taking a good product and making it better, one which the commercial studios will leap at in the same way they did flash - and the studios do dislike flash becuase its not intended to be a TV animation tool.

My point is making the matting external so any layer can self-matte another layer, will make the product very much stronger. It will allow aniamtors and technicians to really push the envelope.

And that will be good for sales.

Rhoel.

Please download the scene amd play with the bouncing ball scene - it's both fun and demonstrates just how flexable an all-in-one layer system works. The x-y axis has just two keys .. the scale axis has just two keys. With those four keyframes, you can put the ball anywhere, fast.

As a test, change the ball positions from rear-left to front-right ... even without understanding how the scene is built, it should take you less than a minute to get it right. In a multi-tasking studio, any compositor should be able to modify a scene without hours of figuring out how something is built: Time is money, and time should be on the creative bit, not swearing at some deeply hidden modification. Having to modify multiple clones of match-moving layers really makes me swear as its un-neccessary, and clunky.

Modifing this scene ins't.
User avatar
Fazek
Posts: 246
Joined: Thu Apr 13, 2006 1:37 pm
Location: Hungary
Contact:

Post by Fazek »

Let's say you have a scene with many layers everywhere. You want to collect a matte information from some layers (to make a common, not-overlapped drop shadow with transparency, for example).

It is possible to write a program to calculate ("render") a new layer with collected and mixed information of many other layers. You can do this statically, so a program goes through the timeline and calculates the required points and shapes of the new layer and inserts keyframes, to exactly copy the actual state. Or in a more dynamic way, you can use a LayerScript to follow the possible later changes in the movement.

Perhaps it is possible to put some new, complex layer-creating tools into Moho. (For example, you select a fill shape and say "mask a layer into this", and a program does every layer-creating, grouping, copying etc. tricks. Maybe some new functons needed in the LUA interface to do this in LUA, I don't know, sorry.

About the masking to the background: duplicate the background (as an overlay, above the original) and make a match-line mask for the duplication. Then put the character between the two background layers. I think every programs work differently...
- - - Fazek
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

I am wondering... just wondering if that script I have been using (bone master layer script) has been a help for me in doing complex masking?

When you talk about having layers all over the place and using them for matting or masking... that is sort of what I am doing.. I think... sort of.

So.. I have static objects which are easy to mask... and animated/boned characters parts of which need to be masked or part of a mask, etc.

So... if I put these parts or copies of parts/layers wherever I need to to create the mask I need, I still just animate one bone layer... it might make it slightly easier for me. Sometimes I spend a lot of time setting up this layer masks than close the group folder thing and just animate... knowing that the masking is done and I don't have to worry about it.

I am willing to admit right now... that most of this discussion of these matting issues is... well... I still don't get it. I may be missing something or it may very well be over my head. ;)

First off I don't do a lot of layer depth motion... stuff. That "z" thing with layers and camera moves? All my projects so far are completely "flat". Everything is animated "by hand".

If I need a specific type of mask I set it up that way ahead of time... maybe I am not doing very complex projects yet. I am willing to admit my work so far may be pretty "simple".

I hope that I haven't muddied the issue too much. It seems clear that there are some issues with masking for complex projects... so any improvements will obviously help everyone eventually.

-vern
Post Reply