code:
proc=subprocess.Popen(['sh',shFile],stderr=subprocess.PIPE,stdout=subprocess.PIPE)
out,err = proc.communicate()
Problem:
shFile in line 1 runs several shell scripts. When I execute shFile from bash, all the shell scripts are executed. However when I execute it using subprocess.Popen, the last shell script is not executed. Interestingly the last line of the shFile is “echo ‘Done'”, which gets executed.
The last 6 lines of shFile are:
sh /export/hierMultiStream_audSpec_direct/tempScripts/decoder__20_eval.sh
sh /export/hierMultiStream_audSpec_direct/tempScripts/decoder__5_eval.sh
sh /export/hierMultiStream_audSpec_direct/tempScripts/decoder__20_eval.sh
sh /export/hierMultiStream_audSpec_direct/tempScripts/decoder__15_eval.sh
sh /export/hierMultiStream_audSpec_direct/tempScripts/decoder__10_eval.sh
echo "Done"
Only decoder__10_eval.sh is not executed. If I change the order of the scripts, once again the last one fails to execute.
subprocess does not do any magic. Note that
shis often not bash on modern Linux systems, but a lightweight shell such as dash. If your shell script is bash-specific, you therefore need to change'sh'to'bash'.In any case, it is extremely unlikely (less likely than winning the lottery ten times in a row) that
sh(orbash) would “forget” to execute a line in a script. Also make sure thatshFileis what you expect.Looking at your updated script, the error is almost certainly in the scripts you evaluate. For example, they might simply run out of temporary files or other inadvertently shared resources. Please post the
decoder__scripts, or reproduce the problem with shFile containing:If the “10” output is missing, then you’ve really got a problem with sh. Otherwise (and again, with extremely high certainty), the problem lies in the tempScripts you’re executing.