It's best to render out a png sequence and then composite or compile the frames in another program. For a variety of reasons (described at the end of this post,) this a common workflow for most animation software, especially when the intent is compositing.
But if you really need to render to a movie file, as Slow Tiger mentions, you need to use a format that supports at least 32-bits (24-bit for RGB and 8-bits for the alpha channel.) There are basically two ways to do this on Windows:
1. If you're using Moho 12.5 and earlier and you are running the x64 version,
make sure you also have the x32 version of Moho installed because this one supports Quicktime. Naturally, you will also need to have Quicktime installed. Now, when you save as a .mov file, even in the x64 version of Moho, you will see the option for Quicktime and the option for Millions+ (the 'plus' is the alpha channel.) When you render from the x64 version, Moho invisibly uses the x32 version to render the .mov file.
Note that this isn't an option with Moho 13 because there is no x32 version available.
2. The other option is to use 'AVI For Windows' (not to be confused with 'AVI'.) After choosing this and clicking OK, another window will pop up that allows you to choose from available codecs. You'll need to choose one that supports an alpha channel. On my computer, I would probably choose something like MagicYUV-RGBA or MagicYUV-YUVA 4444, and it looks like this:
On your computer this will probably look different because it depends on the codec you have installed. In my case, I'm using the commercial version of MagicYUV. If you see PNG listed, that will work too (this is technically the same as rendering a PNG image sequence but in an .avi container.) Basically, you will need to install the codec you wish to use to see it in this list. You'll have to research that bit; I don't normally render movie files out of Moho so I don't have current recommendations for codec packs.
IMO, it really is best and easiest to render a PNG image sequence and then compile the frames in another program. The main reason is because rendering an image sequence uses less computer resources (i.e., processing and RAM) so Moho can render faster and more efficiently. And if Moho happens to crash or renders some mistakes or bad frames, then you only need to re-render a portion of your animation to fix it, not the entire animation from frame 1 as would be the case with .mov or .avi. After the image sequence is done, you can compile it in any editing or compositing program. Many media players can do this too. This is a common workflow for many professional artists and studio productions.
Finally, I recommend using Moho Exporter vs. the basic Export Animation command. Moho Exporter has several options to make it easier to render a scene, and it has the ability to automatically break out a scene for compositing using your settings from the Layer Comps window. (For example, you might use this to render separate passes for character, foreground/midground/background, props, effects, and special matte passes for lighting and shadow effects...all from a single project file. The possibilities are endless!) The typical workflow is to render your frames from Moho, composite in a compositing program (like After Effects, Fusion, HitFilm or Nuke,) and then output a movie file to place in a video editing program. Some video editing programs will let you composite as well but it's easier to do this sort of thing in a dedicated compositing program.
Hope this helps.