google bigquery - QueueDequeue step is a significant bottleneck in Tensorflow code -


i've seen few other questions regarding dequeueing step causing bottleneck have tried many of suggestions without success:

  1. i using multiple threads based on cpu count
  2. i have tried small batches (of 100) , larger batches (of 1000)
  3. i have tried switching shuffle_batch_join , batch_join

none of these things seem helping overall time. extent dequeue step causing bottleneck in code seems worse others have experienced. when @ timeline of other steps practically disappear in comparison. wondering if partially using bigquery , bigqueryreader source although other people experiencing slowdown assuming isn't cause.

timeline

i'm not entirely sure how interpret chart doesn't seem the problem being caused empty queue. tensorboard

additional info

  • capacity set batch size * 10
  • min after dequeue set batch size * 2 + 1
  • enqueue many set true

does have thoughts on else might able try speed things while still using bigquery source of data?


Comments

Post a Comment

Popular posts from this blog

Ansible warning on jinja2 braces on when -

i using code in vars my_var: "{{lookup('env','my_var') | default(true, true)}}" i using like - include: task.yml when: my_var this worked without issue in ansible 2.2 in ansible 2.3 warning this [warning]: when statements should not include jinja2 templating delimiters such {{ }} or {% %}. found: {{lookup('env','my_var') | default(true, true)}} so if use my_var: "lookup('env','my_var') | default(true, true)" , mean remove {{ }} , no warning my_var evaluated true no matter if pass false env variable. works fine in previous code gave how can fix it? the cleanest way set fact instead of using template in variable (it prevent lookup plugin being called multiple times in included tasks - not huge performance gain, cleaner): - set_fact: my_var: "{{lookup('env','my_var') | default(true, true)}}" also mind evaluate true in cases when environment my_var diff...
Read more

Parsing a protocol message from Go by Java -

i write server(go)-client(java) programe, , use protobuf communication. define proto file , share between server , client. in server side: compile shared proto file go protoc serialize object proto.marshal send client make request service in client side: compile shared proto file java protoc get bytes transferred via http deserialize bytes object. here following error: "com.google.protobuf.invalidprotocolbufferexception: while parsing protocol message, input ended unexpectedly in middle of field. mean either input has been truncated or embedded message misreported own length." i confirmed http work fine , value of bytes received in client side same bytes send server. have same problem on this? here proto file syntax = "proto3"; package tutorial; message person { string name = 1; int32 id = 2; string email = 3; enum phonetype { mobile = 0; home = 1; work = 2; } message phonenumber { ...
Read more

node.js - Node js - Trying to send POST request, but it is not loading javascript content -

i have been trying send post request website, , has not been loading full site. i guessing problem node js not loading javascript content, executing it. can see javascript in response, though never loads content javascript files producing in html. what mean is, doesn't execute javascript. how make node js code overcome this, , wait the full website load? have tried native code, , "request' module, both producing same errors. if helps, don't need native javascript. modules fine. some code have tried (request module): var options = { method: 'post', uri: 'https://url', formdata: { 'email': 'email', 'prize': '0', 'transactional': 'on' }, headers: { /* 'content-type': 'application/x-www-form-urlencoded' */ // set automatically } }; rp(options) .then(function (body) { console.log(body) }) .catch(function (err) { console.log(err) }); when fetch content...
Read more