Question

Basically I am running mapreduce so I’m going to pipe in to the reducer. What I’m having trouble with is how to specify a directory path which I will use with os.listdir to essentially get to and then compute all the files in that directory alone. But I know I only want what is in the last directory and not the sub-directories on that same level. I may consider also using fileinput().

I think I may be using stdin wrong here, depends on if it has to be from the command line or if it can be indirectly from the command line.

This was my command-line input: “./path1/path2/path3” ./map.py | sort | ./red.py

What I got as an error from doing what I want to do is -bash, not a directory ./pythonfile.py

Using Python 2.7.2

This is what I’m doing:

def func():
    path = sys.argv[0]
    return [filenames for filenames in os.listdir(path)if os.path.isfile(os.path.join(path,filenames))]

if func() is not None: 
   for file in func():
       sys.stdin.read()
       ...etc..

Answer 1

"./path1/path2/path3" ./map.py | sort | ./red.py

That tells bash to execute the program ./path1/path2/path3 with the argument ./map.py. This is almost certainly not what you want to do.

path = sys.argv[0]

This is the name of the script — the first element of argv[] is, by convention, the name of the program that is being executed. See this:

$ cat echo.py 
#!/usr/bin/python
import sys
print sys.argv[0]
$ chmod 755 echo.py
$ ./echo.py
./echo.py
$ 

Perhaps what you intended to do was echo the string to the script:

echo path1/path2/path3 | ./map.py | sort | ./red.py

But then you could not use the argv[] array to get at it easily. In fact, it is very difficult to properly parse multiple pathnames from a free-form input like this, so I’d avoid it entirely, and pass the names as arguments:

./map.py path1/path2/path3 /a/s/econd/pathname

You could iterate over all paths passed as arguments using something like this:

>>> a=["nothing", "passwd", "shadow"]
>>> [f for f in a[1:]]
['passwd', 'shadow']
>>> import os
>>> [f for f in a[1:] if os.path.isfile(f)]
['passwd', 'shadow']
>>>