Running builder (content generator) functions in subprocesses on Windows

- Refactored all builder (make_*) functions into separate Python modules along to the build tree
- Introduced utility function to wrap all invocations on Windows, but does not change it elsewhere
- Introduced stub to use the builders module as a stand alone script and invoke a selected function

There is a problem with file handles related to writing generated content (*.gen.h and *.gen.cpp)
on Windows, which randomly causes a SHARING VIOLATION error to the compiler resulting in flaky
builds. Running all such content generators in a new subprocess instead of directly inside the
build script works around the issue.

Yes, I tried the multiprocessing module. It did not work due to conflict with SCons on cPickle.
Suggested workaround did not fully work either.

Using the run_in_subprocess wrapper on osx and x11 platforms as well for consistency. In case of
running a cross-compilation on Windows they would still be used, but likely it will not happen
in practice. What counts is that the build itself is running on which platform, not the target
platform.

Some generated files are written directly in an SConstruct or SCsub file, before the parallel build starts. They don't need to be written in a subprocess, apparently, so I left them as is.

1 files changed, 4 insertions, 86 deletions

diff --git a/editor/icons/SCsub b/editor/icons/SCsub
index 7f94073e01..31bf8f116a 100644
--- a/editor/icons/SCsub
+++ b/editor/icons/SCsub
@@ -1,96 +1,14 @@
 #!/usr/bin/env python
 
 Import('env')
-from compat import StringIO
+from platform_methods import run_in_subprocess
+import editor_icons_builders
 
-def make_editor_icons_action(target, source, env):
 
-    import os
-
-    dst = target[0].srcnode().abspath
-    svg_icons = source
-
-    icons_string = StringIO()
-
-    for f in svg_icons:
-
-        fname = str(f)
-
-        icons_string.write('\t"')
-
-        with open(fname, 'rb') as svgf:
-            b = svgf.read(1)
-            while(len(b) == 1):
-                icons_string.write("\\" + str(hex(ord(b)))[1:])
-                b = svgf.read(1)
-
-
-        icons_string.write('"')
-        if fname != svg_icons[-1]:
-            icons_string.write(",")
-        icons_string.write('\n')
-
-    s = StringIO()
-    s.write("/* THIS FILE IS GENERATED DO NOT EDIT */\n")
-    s.write("#ifndef _EDITOR_ICONS_H\n")
-    s.write("#define _EDITOR_ICONS_H\n")
-    s.write("static const int editor_icons_count = {};\n".format(len(svg_icons)))
-    s.write("static const char *editor_icons_sources[] = {\n")
-    s.write(icons_string.getvalue())
-    s.write('};\n\n')
-    s.write("static const char *editor_icons_names[] = {\n")
-
-    # this is used to store the indices of thumbnail icons
-    thumb_medium_indices = [];
-    thumb_big_indices = [];
-    index = 0
-    for f in svg_icons:
-
-        fname = str(f)
-
-        icon_name = os.path.basename(fname)[5:-4].title().replace("_", "")
-        # some special cases
-        if icon_name in ['Int', 'Bool', 'Float']:
-            icon_name = icon_name.lower()
-        if icon_name.endswith("MediumThumb"):  # don't know a better way to handle this
-            thumb_medium_indices.append(str(index))
-        if icon_name.endswith("BigThumb"):  # don't know a better way to handle this
-            thumb_big_indices.append(str(index))
-
-        s.write('\t"{0}"'.format(icon_name))
-
-        if fname != svg_icons[-1]:
-            s.write(",")
-        s.write('\n')
-
-        index += 1
-
-    s.write('};\n')
-
-    if thumb_medium_indices:
-        s.write("\n\n")
-        s.write("static const int editor_md_thumbs_count = {};\n".format(len(thumb_medium_indices)))
-        s.write("static const int editor_md_thumbs_indices[] = {")
-        s.write(", ".join(thumb_medium_indices))
-        s.write("};\n")
-    if thumb_big_indices:
-        s.write("\n\n")
-        s.write("static const int editor_bg_thumbs_count = {};\n".format(len(thumb_big_indices)))
-        s.write("static const int editor_bg_thumbs_indices[] = {")
-        s.write(", ".join(thumb_big_indices))
-        s.write("};\n")
-
-    s.write("#endif\n")
-
-    with open(dst, "w") as f:
-        f.write(s.getvalue())
-
-    s.close()
-    icons_string.close()
-
-make_editor_icons_builder = Builder(action=make_editor_icons_action,
+make_editor_icons_builder = Builder(action=run_in_subprocess(editor_icons_builders.make_editor_icons_action),
                                     suffix='.h',
                                     src_suffix='.svg')
+
 env['BUILDERS']['MakeEditorIconsBuilder'] = make_editor_icons_builder
 env.Alias('editor_icons', [env.MakeEditorIconsBuilder('#editor/editor_icons.gen.h', Glob("*.svg"))])