Question

I’m trying to make portable pyDev automatically set up the python interpreter by using a batch script that’s run before eclipse launches. I came across the following bash-script for doing just that, but I’m unable to translate it to batch (which I’m familiar with):

PYTHONPATH=/my/extra/python/modules python /my/eclipse/plugins/org.python.pydev_*/PySrc/interpreterInfo.py 2>/dev/null | sed 's/INS_PATH$//g;s/OUT_PATH$//g;s/^EXECUTABLE:/Executable\\:/g' | tr -d '\n' | cat <(echo -en 'eclipse.preferences.version=1\nINTERPRETER_PATH_NEW=Name\:python\:EndName\:') - <(echo '&&&&&') >/my/workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs

I tried breaking it down to see what’s being replaced/piped where, but it doesn’t make sense:

PYTHONPATH= App/Eclipse/plugins/org.python.pydev_*/PySrc/interpreterInfo.py 2> /dev/null 
|sed '  s/ INS_PATH$/ / g; REM Delete all instances of INS_PATH that are at the end of a line
        s/ OUT_PATH$/ / g; REM Delete all instances of OUT_PATH that are at the end of a line
        s/ ^EXECUTABLE:/ Executable\\:/ g REM Replace all instances of "\nEXECUTABLE:" that are at the beginning of a line with "Executable\:"
' 
|tr -d '\n' 
|cat <(echo -en 'eclipse.preferences.version= 1 \n INTERPRETER_PATH_NEW= Name\:python\:EndName\:') - <(echo '&&&&&') 
>/my/workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs

---

One of the most confusing parts is the the strange use of colons and use of ‘/’ vs ‘\’.
Looking in config files I see osgi.framework=file\:plugins/org.eclipse.osgi_3.8.0.v20120529-1548.jar

Answer 1

I’ve tried to format the “original” to make it more readable. I’ll try to explain what’s happening from there:

PYTHONPATH=/my/extra/python/modules \
    python /my/eclipse/plugins/org.python.pydev_*/PySrc/interpreterInfo.py 2>/dev/null |
    sed '
        s/INS_PATH$//g;
        s/OUT_PATH$//g;
        s/^EXECUTABLE:/Executable\\:/g
    ' |
    tr -d '\n' |
    cat \
        <(
            echo -en 'eclipse.preferences.version=1\nINTERPRETER_PATH_NEW=Name\:python\:EndName\:'
        ) \
        - \
        <(echo '&&&&&') \
     >/my/workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs

Now, let’s look at this section-by-section to make sense of it. I’m going to go over everything, including those parts which I expect you already know about, for the sake of completeness.

PYTHONPATH=/my/extra/python/modules \

This is setting the variable PYTHONPATH, which, from context, I expect tells python where to look for modules. Because we’re specifying it within the same “line” as a command, two things happen:

1. The variable is automatically exported to the environment
2. The variable is only set for the individual command which is being run (ie: just the “first” command, not the whole pipeline, and it won’t be in the shell’s environment for use by later commands)

Next, the main command:

 python /my/eclipse/plugins/org.python.pydev_*/PySrc/interpreterInfo.py 2>/dev/null |

Here the “python” interpreter is run, and passed the interpreterInfo.py script to execute.

The _* means that we’re not specifying the specific version of org.python.pydev which we want (it is assumed that only a single version will be found, and that all versions will behave similarly).

The 2>/dev/null means that any output on stderr, where errors and debug info is normally piped, will be ignored (piped to nowhere).

Now the meat of it:

sed '
    s/INS_PATH$//g;
    s/OUT_PATH$//g;
    s/^EXECUTABLE:/Executable\\:/g
' |

This does three things. The first two are similar: INS_PATH and OUT_PATH are removed from the ends of lines. Next, and the first part which you were asking about, : characters are replaced with \:. Why the double \\ if \: is the goal? Mostly this is due to pedantry. Though \: is not a valid escape sequence, and so most seds will interpret it as a literal \ marks the beginning of an Escape Sequence, so the canonical way of specifying that you want sed to output a literal \ is to specify it as \\. Hence, for the sequence \:, it is specified as \\:.

The /g at the end of each of these substitution commands means “replace globally”, ie: every time the match is found, not just the first time.

The next line is straightforward:

tr -d '\n' |

This removes all newlines in the output. If I understand correctly, this results in a \: separated list of paths, on a single line.

Back to parts which may require explanation:

cat \
    <(
        echo -en 'eclipse.preferences.version=1\nINTERPRETER_PATH_NEW=Name\:python\:EndName\:'
    ) \
    - \
    <(echo '&&&&&') \
 >/my/workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs

That’s a lot to go through all at once. Let’s summarise:

cat input-one input-two input-n > output

cat combines (“concatenates”) multiple inputs into a single output. This is actually the intended use of the cat command, though you may see it much more often used to output a single input, eg: cat input-one.

<( some-command )

Runs some-command and redirects its output into a File Descriptor, whose path (/dev/fd/N) is passed used as an argument in the context which <( ... ) was originally in. For example: cat <( echo foo ) would run a command similar to cat /dev/fd/64, and echo foo, redirecting the output of echo foo to file descriptor 64, for reading by cat. The output is foo, just as it would be if you ran echo directly.

Now, combine these two things:

cat <( some-command ) some-input <( some-other-command )

This will combine the output of some-command, the content of some-input, and the output of some-other-command, into a single stream. In the script you’re working with, these are all being combined into a single output file.

So what about the content that is actually being build for this output? Let’s look at that line-by-line as well:

<(
    echo -en 'eclipse.preferences.version=1\nINTERPRETER_PATH_NEW=Name\:python\:EndName\:'
)

This outputs the literal:

eclipse.preferences.version=1
INTERPRETER_PATH_NEW=Name\:python\:EndName\:

echo -e means “interpret backslash escape sequences”, so \n expands into a newline. echo -n means “do not end output with a newline”, so whatever comes next is part of the same line. In this case, that means that whatever is concatenated next will become part of the INTERPRETER_PATH_NEW= line.

-

A - is shorthand for “read from standard input”, which is the default if you run cat by itself. Many commands use this shorthand, though for those that don’t, another way of specifying this is /dev/stdin. It means “read in what was piped to this command”, ie: the output from tr, which itself defaulted to reading the output from sed, which defaulted to reading the output from python.

<(echo '&&&&&')

This ends the stream with five & characters and a newline. I don’t actually know the reason for this, so perhaps someone can comment to fill me in.

>/my/workspace/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.python.pydev.prefs    

As a final step, all output is redirected (via the > symbol) to a file, org.python.pydev.prefs.

With that big overview of what is happening out of the way, it’s now time to answer the pesky details which you were asking about:

Why all the \ characters?

Well, in the same way that we escaped \ within sed, we’re also escaping : within the .prefs file. \: in the .prefs file seems to mean “a literal :” in the same way that \\ within sed specifies a literal \.

What about the / characters?

/ is the directory separater in UNIX systems (including modern Mac OSs, ie OSX). On Windows systems, \ is used. In many languages, as noted above, \ is also used as an escape character, so to specify a literal \, you may need to write it as \\. There is no hard rule for this. Some cross-platform languages, aware of the problem, allow / to be used as a directory separator on Windows as well.

I hope that clears things up. Let me know if there is anything else which requires explanation, and I’ll try to edit my answer to add additional information.