multithreading - Python explicit next() slower than a for loop in multithreaded producer-consumer paradigm -


suppose have following code runs 10 iterations , calls generator gen_next_img_batch:

for _ in get_next_img_batch(train_data_paths, classes, batch_size):      pass 

now, suppose replace following, equivalent (to mind) iteration:

for in range(10):     next(get_next_img_batch(train_data_paths, classes, batch_size)) 

the conundrum have first snippet takes 17 seconds, while second takes 42 seconds. moreover, every iteration of second snippet takes more time previous iteration. is, iteration i+1 of second snippet takes longer iteration i. iterations of first snippet take same time.

an important fact seems generator gen_next_img_batch iterated on takes data python queue populated in background thread. more specifically, in background, separate thread loading data queue, , every call next() yields object queue. when asynchronous behaviour omitted, both snippets take same time.

they not equivalent. specifically, first fragment calls get_next_img_batch() once, while second calls function 10 times. equivalent fragment might be:

it = get_next_img_batch(train_data_paths, classes, batch_size)) in range(10):     next(it) 

Comments

Popular posts from this blog

node.js - Node js - Trying to send POST request, but it is not loading javascript content -

javascript - Replicate keyboard event with html button -

javascript - Web audio api 5.1 surround example not working in firefox -