Render.exe subprocess killing process group ends current script

I'm loading Maya's Render.exe on our farm with a subprocess:
*p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)*

Pre-2022/23/Py3 (aka the Py2.7 days) I was pushing the line by line output to stdout through:
*sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)*

but with the switch to Py3, this no longer works. I've looked for various solutions online and have not found one that properly works for me.

Examples of failed attempts and why:

*os.environ["PYTHONUNBUFFERED"] = "1"*
this in and of itself isn't outputting anything to stdout, though perhaps I just need to then add a line to push something to stdout. but what will work?

*sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True)*
fails on the farm

Does anyone have a working solution for this?  It seems like it must be super common as you obviously don't want to dump the whole thing at once after its all over. Also we are using iRush and without it being able to read it as it happens, we are unable to determine the success of the render at that point.
Thanks!

IGNORE ABOVE:
EDIT:

This was helpful. It showed me that the original issue was not even what I thought it was! it has something to do with this line failing to kill the process after the render fails. Seemed to work fine before I updated the scripts to py3 compatibility (but attempting to make sure it was still compatible with py2.7.
To be honest though, this seems like it shouldnt work to me though. Its killing the python process that is running the script itself, so how did it ever get to the sys.exit(1) part?!..

def failed_render(pid=None):
    """Ends render script with 'fail' status.
    sys code: 0=Done, 1=Failed, 2=Retry
    """
    if pid:
        # line below is the last thing executed.... how could it  get past this if this script is running as a subprocess of the process that is running the script itself?
        os.killpg(os.getpgid(pid), signal.SIGKILL)
    sys.exit(1)

The pid is correct.
the os.getpgid is correct.
but when it gets to this line it completely crashes out without closing out of the process.

when i query the process associated with the group (result of os.getpgid) its the python process itself! I can’t remember for the life of me what the process was previously but that can’t be what it was before because if it was, then killing the group would kill the python script itself! so of course it can’t exit cleanly.

Is there some fundamental difference in how subprocess.Popen(cmd) is connecting?

cmd= [os.environ["MAYA_LOCATION"] + "/bin/Render",
                '-renderer', 'arnold',
                '-proj', opts.project_path,
                 '-preRender', pre-cmd,
                opts.file]

when I query with psutil the process and the group process give (hashed out numbers, shortened CMDLINE of the second):

psutil.Process <PID:#####; PPID:#####; NAME:'Render'; PATH:''; CMDLINE:[]; UID:###; GID:###;>
psutil.Process <PID:#####; PPID:####; NAME:'python2.7'; PATH:'/usr/bin'; CMDLINE:['python', '//NetworkPath/dir/renderScript.py']

Try this:

cmd = "Render.exe"
process = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)

while process.poll() is None:
   stdout_line = str(process.stdout.readline())
   print(stdout_line)

process.stdout.close()
1 Like