Sign up ×

it's a similar source I've uploaded yesterday but solve some problems thanks to your help.

it's a little bit shame asking another question about somewhat same problem:(

but though I deliberated what is the problem all day,I failed to find. So,

it looks good and do work,but the problem is, some thread never terminate themselves for a long time.

I waited even 10 minutes but 6 threads are still alive.

it's the biggest mystery thing since I started learing programming...

would you please teach me what's wrong with it?

import os 
import threading 
import multiprocessing 

def finder(path, q, done): 
    for root, dirs, files in os.walk(unicode(path)): 
        for dirname in dirs: 
            if target in dirname.lower(): 
        for name in files: 
            if target in name.lower(): 
    #print "good bye",threading.current_thread()
    #print threading.active_count()

def printer(q,done,worker_count): 
    total = 0 
    while 1: 
        try: done.get_nowait() 
        except: pass 
        else: total += 1 

        try: tmp=q.get(timeout=1) 
        except: pass 
        else: print tmp            

        if total == worker_count: 

if __name__ =="__main__": 

    results = multiprocessing.Queue() 
    done = multiprocessing.Queue() 

    root, dirs, files = os.walk(u"C:\\").next() 
    for dirname in dirs: 
        if target in dirname.lower(): 
    for name in files: 
        if target in name.lower(): 

    target=raw_input("what you wanna get\n") 

    for i in xrange(thnum): 
        full_path = os.path.join(root, dirs[i]) 
        t=threading.Thread(target=finder,args=(full_path, results, done,)) 


share|improve this question
It could be that it really takes that much longer. For example large cache folders from mail clients like thunderbird or outlook express, and from web browsers, may make this take very long (????) Or you could be looking at cycles somehow. Can you build in code somehow to signal the existing threads that they should print out their status? –  Andre Blum Jul 10 '12 at 0:26
by comparing the result with that of my another program, it just find all the file but the thread just stop but not exit... –  from __future__ Jul 10 '12 at 0:37
Um… you're walking the filesystem and putting every file in the tree into the queue, and then for each file in the queue you're walking the entire tree again and pushing every file in that tree into the queue. All this pointless repetition is going to take a very, very long time for even a modest subtree. –  abarnert Jul 10 '12 at 1:13
Actually, hold on a sec… you're using target without ever defining it. So, unless your C: drive is completely empty, this is obviously not the code you're running, because this code will throw a NameError and quit immediately. –  abarnert Jul 10 '12 at 1:17
can you try and run in a shallower and overseeable subdirectory, instead of your hard disk's root? –  Andre Blum Jul 10 '12 at 2:09

1 Answer 1

up vote 0 down vote accepted

[ First, of course, I had to make some trivial changes to make this code 'compile']

The good news is: it just works as you think it should work. Well done. The bad news is: it doesn't work as fast as you had expected.

On my machine, running it on my home directory alone already takes approx 10 minutes:

[andre@hp ~]$ time python
what you wanna get
... results removed ...

real        9m39.083s
user        0m30.368s
sys         0m22.664s
[andre@hp ~]$

The question is whether implementing this in processes and threads is a good idea. I think not. Chances are the performance suffers from this massive multithreading.

share|improve this answer
I finally make it work with coroutine. make a connection with coroutine instead of using "done" with Queue.put, but it just 2 seconds faster:( I agree with you. it was not a good idea. thank you! –  from __future__ Jul 11 '12 at 6:23

Your Answer


