Take the 2-minute tour ×

I want to count number of files recursively in a remote server path using rsync in Python? I tried it doing like this:

def find_remote_files(source, password):
    cmdline = ['sshpass', '-p', password, 'rsync', '--recursive', source]
    with open(os.devnull, "w") as devnull:
        proc = subprocess.Popen(cmdline, stdout=subprocess.PIPE, stderr=devnull)
            for entry in proc.stdout:
                items = entry.strip().split(None, 4)
                if not items[0].startswith("d"):
                    yield lent(items[4])
            # On any exception, terminate process and re-raise exception.

It works in a situation where I have less files. But in case if I have more than 3000 files the rsync will take a long time to store it in a list and again calculate the length. That's why, I wanted to know if there is a rsync command just to count the file.

share|improve this question
add comment

1 Answer

up vote 0 down vote accepted

I would use a different approach using fabric, which is a great tool for executing remote commands.

from fabric.api import run, env  
env.host_string = 'example.org'
output = run('find /tmp -type f | wc -l')
num_files = int(output)

Now you have the number of files in your variable num_files. I was just using the find command to search for files recursively beginning at directory /tmp, and counted the returned lines with wc -l.

share|improve this answer
Where should I put the username and password of the host_string = 'example.org'? –  user2545177 Jul 9 '13 at 8:34
just check out the documentation on fabric: docs.fabfile.org/en/1.6 - you can provide host and user on the command line or in your fabfile. The password should not be used, since the normal approach is by using ssh keys without password. –  mawimawi Jul 9 '13 at 11:04
add comment

Your Answer


