Moho Animation Forum

Windows Unicode Path Issue with io.open on Non-English Systems (e.g., Japanese Windows) in Moho Pro 14 & Workaround Proposal

Hi everyone,

I would like to report a critical and long-standing issue regarding Lua scripting on Windows when dealing with file/folder paths that contain non-ASCII characters (such as Japanese Kanji, Hiragana, Katakana, accents, or emojis).

As we develop more advanced data-driven scripts and tools for Moho Pro 14 (which runs Lua 5.4.4), this issue has become a severe bottleneck for international users.

1. The Problem (Technical Context)
On macOS and Linux, the operating system natively uses UTF-8 for file system paths, allowing Lua’s standard io.open to work flawlessly with any language [cite: 2, 19].

However, on Windows, the OS expects either:

ANSI/Narrow APIs: Uses local codepages (e.g., CP932 for Japanese Windows, CP936 for Chinese Windows) [cite: 4, 15].

Wide APIs: Uses UTF-16 (wchar_t) to support full Unicode [cite: 3, 4].

Because the unmodified standard Lua library compiles against ANSI C, standard functions like io.open only support OS-native ANSI narrow strings on Windows [cite: 4, 15]. When a Moho script passes a UTF-8 path (which Moho internally uses) containing Japanese characters to io.open, Windows tries to interpret it using the local system codepage (CP932), resulting in invalid character parsing, "file not found" errors, or silent failures [cite: 4, 5, 15].

2. Real-World Impact
This prevents Japanese (and other multilingual) animators from using any script that reads or writes external assets (such as CSV motion files, JSON configurations, or custom images/textures) if:

Their Windows username contains Kanji or Kana (which is extremely common as Windows often defaults to the user's real name for the user folder: C:\Users\坂本\AppData\...).

Their active Moho project file is saved in a folder with Japanese characters.

Their Moho Custom Content Folder path contains non-ASCII characters [cite: 2, 12].

3. Our Painful Workaround (The PowerShell Bypass)
To prevent our tool scripts from crashing for worldwide users, we had to write an expensive and complex fallback mechanism. Since we cannot rely on io.open directly on Windows, and using io.popen in a GUI Windows application has a notorious bug that can freeze the parent app indefinitely [cite: 3, 27], we had to bypass the OS by spinning up a background PowerShell process just to copy a text file to an ASCII-safe directory before reading it.

Here is the fallback code we implemented in our tool:

Lua
local _safeReadCounter = 0
local function SafeReadText(filePath)
if not filePath or filePath == "" then return nil end

local isWindows = (package.config:sub(1,1) == "\\")

if isWindows then
filePath = string.gsub(filePath, "/", "\\")
end

-- Attempt standard read first (usually succeeds on macOS/Linux)
local f = io.open(filePath, "r")
if f then
local content = f:read("*a")
f:close()
return content
end

-- Windows Fallback: Bypass path garbling via PowerShell
if isWindows then
local tmpDir = os.getenv("TEMP") or "C:\\Temp"

_safeReadCounter = _safeReadCounter + 1
local rID = tostring(os.time()).. "_".. tostring(_safeReadCounter)

local tmpPathFile = tmpDir.. "\\rw_path_".. rID.. ".txt"
local tmpDataFile = tmpDir.. "\\rw_data_".. rID.. ".txt"

local fTmp = io.open(tmpPathFile, "wb")
if fTmp then
fTmp:write(filePath)
fTmp:close()

-- Call PowerShell silently to resolve the UTF-8 literal path and copy it to an ASCII temporary file
local psScript = string.format('powershell -WindowStyle Hidden -NoProfile -ExecutionPolicy Bypass -Command "$p = Get-Content -LiteralPath \'%s\' -Encoding UTF8; Copy-Item -LiteralPath $p -Destination \'%s\' -Force"', tmpPathFile, tmpDataFile)
os.execute(psScript)

local fData = io.open(tmpDataFile, "rb")
if fData then
local content = fData:read("*a")
fData:close()

-- Clean up temp files
os.remove(tmpPathFile)
os.remove(tmpDataFile)

if content and content ~= "" then
return content
end
end
os.remove(tmpPathFile)
os.remove(tmpDataFile)
end
end

return nil
end
Spawning a hidden shell command (os.execute) just to parse/read a basic text configuration file is extremely heavy, adds performance overhead, and triggers security warnings on some machines. However, it is currently the only pure-Lua way to bypass the standard io.open ANSI limitation on Windows without crashing Moho or freezing the UI [cite: 3, 27].

4. A Proposal to the Developers (Lost Marble)
To solve this once and for all, we kindly request the development team to implement one of the following native solutions:

Expose a Native Unicode File API: Provide a built-in stream/file API under the MOHO or LM namespace (e.g., MOHO.OpenFileUTF8 or LM.ReadFile) that internally handles UTF-8 to UTF-16 translation and calls the Win32 Wide APIs (_wfopen_s or CreateFileW) [cite: 3, 4, 27].

Patch the Lua DLL with UTF-8 Wrappers on Windows: Many game engines and embedded platforms patch the native Lua binaries on Windows to intercept file system calls (like fopen, popen, remove, rename) and automatically perform the UTF-8/UTF-16 wide conversion under the hood (for example, the open-source lua-unicode project wrapper) [cite: 17, 49].

Having native, Unicode-safe file handling inside Moho would dramatically improve script stability and ensure a seamless experience for animators in Japan, China, Korea, Europe, and everywhere else in the world.

Thank you for your consideration, and keep up the amazing work with Moho 14!