multithreading - How can multiple threads share an iterator? -

i've been working on function copy bunch of files source destination using rust , threads. i'm getting trouble making threads share iterator. not still used borrowing system:

extern crate libc; extern crate num_cpus;  use libc::{c_char, size_t}; use std::thread; use std::fs::copy;  fn python_str_array_2_str_vec<t, u, v>(_: t, _: u) -> v {     unimplemented!() }  #[no_mangle] pub extern "c" fn copyfiles(     sources: *const *const c_char,     destinies: *const *const c_char,     array_len: size_t, ) {     let src: vec<&str> = python_str_array_2_str_vec(sources, array_len);     let dst: vec<&str> = python_str_array_2_str_vec(destinies, array_len);     let mut iter = src.iter().zip(dst);     let num_threads = num_cpus::get();     let threads = (0..num_threads).map(|_| {         thread::spawn(|| while let some((s, d)) = iter.next() {             copy(s, d);         })     });     t in threads {         t.join();     } }  fn main() {} 

i'm getting compilation error have not been able solve:

error[e0597]: `src` not live long enough   --> src/main.rs:20:20    | 20 |     let mut iter = src.iter().zip(dst);    |                    ^^^ not live long enough ... 30 | }    | - borrowed value lives until here    |    = note: borrowed value must valid static lifetime...  error[e0373]: closure may outlive current function, borrows `**iter`, owned current function   --> src/main.rs:23:23    | 23 |         thread::spawn(|| while let some((s, d)) = iter.next() {    |                       ^^                          ---- `**iter` borrowed here    |                       |    |                       may outlive borrowed value `**iter`    | help: force closure take ownership of `**iter` (and other referenced variables), use `move` keyword, shown:    |         thread::spawn(move || while let some((s, d)) = iter.next() { 

i've seen following questions already:

value not live long enough when using multiple threads i'm not using chunks, try share iterator through threads although creating chunks pass them threads classic solution.

unable send &str between threads because not live long enough i've seen of answers use channels communicate threads, i'm not quite sure using them. there should easier way of sharing 1 object through threads.

why doesn't local variable live long enough thread::scoped got attention, scoped supposed fix error, since in unstable channel see if there way of doing using spawn.

can explain how should fix lifetimes iterator can accessed threads?

here's mcve of problem:

use std::thread;  fn main() {     let src = vec!["one"];     let dst = vec!["two"];     let mut iter = src.iter().zip(dst);     thread::spawn(|| {         while let some((s, d)) = iter.next() {             println!("{} -> {}", s, d);         }     }); } 

there multiple related problems:

1. the iterator lives on stack , thread's closure takes reference it.
2. the closure takes mutable reference iterator.
3. the iterator has reference vec lives on stack.
4. the vec has references string slices likely live on stack not guaranteed live longer thread either way.

said way, rust compiler has stopped executing 4 separate pieces of memory unsafety.

a main thing recognize thread spawn might outlive place spawned it. if call join right away, compiler cannot statically verify happen, has take conservative path. point of scoped threads — guarantee thread exits before stack frame started in.

additionally, attempting use mutable reference in multiple concurrent threads. there's zero guarantee iterator (or of iterators built on) can safely called in parallel. it's entirely possible 2 threads call next @ exactly same time. 2 pieces of code run in parallel , write same memory address. 1 thread writes half of data , other thread writes other half, , program crashes @ arbitrary point in future.

using tool crossbeam, code like:

extern crate crossbeam;  fn main() {     let src = vec!["one"];     let dst = vec!["two"];      let mut iter = src.iter().zip(dst);     while let some((s, d)) = iter.next() {         crossbeam::scope(|scope| {             scope.spawn(|| {                 println!("{} -> {}", s, d);             });         });     } } 

as mentioned, spawn 1 thread @ time, waiting finish. alternative more parallelism (the usual point of exercise) interchange calls next , spawn. requires transferring ownership of s , d thread via move keyword:

extern crate crossbeam;  fn main() {     let src = vec!["one", "alpha"];     let dst = vec!["two", "beta"];      let mut iter = src.iter().zip(dst);     crossbeam::scope(|scope| {         while let some((s, d)) = iter.next() {             scope.spawn(move || {                 println!("{} -> {}", s, d);             });         }     }); } 

if add sleep call inside spawn, can see threads run in parallel.

i'd have written using for loop, however:

let iter = src.iter().zip(dst); crossbeam::scope(|scope| {     (s, d) in iter {         scope.spawn(move || {             println!("{} -> {}", s, d);         });     } }); 

in end, iterator exercised on current thread, , each value returned iterator handed off new thread. new threads guaranteed exit before captured references.

you may interested in rayon, crate allows easy parallelization of types of iterators.

see also:

• lifetime troubles sharing references between threads
• how use static lifetimes threads?
• thread references require static lifetime?
• lifetime woes when using threads
• cannot call function in spawned thread because "does not fulfill required lifetime"