azure documentdb - Cosmosdb store procedure get less documents than real -


i preparing store procedure on cosmosdb javascript, however, gets less documents real number of documents in collection.

the sproc called c#, c# pass parameter "transmittermmsi" partition key of collection.

first, following query executed in sproc:

var query = 'select count(1) num aisdata a.transmittermmsi="' + transmittermmsi + '"'; 

the result output in response, , value 5761, same real number of documents in collection.

however, when change query following:

var query = 'select * aisdata a.transmittermmsi="' + transmittermmsi + '"'; 

the documents.length output 5574, smaller real number.

i have changed pagesize: -1, should mean unlimited.

i did search google , stack overflow, seems continuation can help. however, tried examples, , don't work.

anyone familiar can help?

the following list scripts.

the sproc js script here, file "downsampling.js" used in c# code:

function downsampling(transmittermmsi, interval) { var context = getcontext(); var collection = context.getcollection(); var response = context.getresponse(); var receivertime; var temptime; var groupkey; var agggroup = new object();  var query = 'select * aisdata a.transmittermmsi="' + transmittermmsi + '"'; var accept = collection.querydocuments(collection.getselflink(), query, { pagesize: -1},     function (err, documents, responseoptions) {         if (err) throw new error("error" + err.message);          // find smallest deviation comparting intervaltime in each group         (i = 0; < documents.length; i++) {             receivertime = date.parse(documents[i].receivertime);             temptime = receivertime / 1000 + interval / 2;             documents[i].intervaltime = (temptime - temptime % interval) * 1000;             documents[i].deviation = math.abs(receivertime - documents[i].intervaltime);              // generate group key each group, combinated of transmittermmsi , intervaltime             groupkey = documents[i].intervaltime.tostring();             if (typeof agggroup[groupkey] === 'undefined' || agggroup[groupkey] > documents[i].deviation) {                 agggroup[groupkey] = documents[i].deviation;             }         }          // tag downsampling         (i = 0; < documents.length; i++) {             groupkey = documents[i].intervaltime;             if (agggroup[groupkey] == documents[i].deviation) {                 documents[i].downsamplingtag = 1;             } else {                 documents[i].downsamplingtag = 0;             }              // remove items not used             delete documents[i].intervaltime;             delete documents[i].deviation;              // replace document             var acceptdoc = collection.replacedocument(documents[i]._self, documents[i], {},                 function (errdoc, docreplaced) {                     if (errdoc) {                         throw new error("update documents error:" + errdoc.message);                     }                 });             if (!acceptdoc) {                 throw "update documents not accepted, abort ";             }          }          response.setbody(documents.length);     }); if (!accept) {     throw new error("the stored procedure timed out."); } }  

and c# code here:

    private async task downsampling()     {         database database = this.client.createdatabasequery().where(db => db.id == databaseid).toarray().firstordefault();         documentcollection collection = this.client.createdocumentcollectionquery(database.selflink).where(c => c.id == aistestcollectionid).toarray().firstordefault();          string scriptfilename = @"..\..\storedprocedures\downsampling.js";         string scriptid = path.getfilenamewithoutextension(scriptfilename);          var sproc = new storedprocedure         {             id = scriptid,             body = file.readalltext(scriptfilename)         };          await trydeletestoredprocedure(collection.selflink, sproc.id);         sproc = await this.client.createstoredprocedureasync(collection.selflink, sproc);          iqueryable<dynamic> query = this.client.createdocumentquery(             urifactory.createdocumentcollectionuri(databaseid, aistestcollectionid),             new sqlqueryspec()             {                 //querytext = "select a.transmittermmsi " + aistestcollectionid + " a",                 querytext = "select a.transmittermmsi " + aistestcollectionid + " a.transmittermmsi=\"219633000\"",             }, new feedoptions { maxitemcount = -1, enablecrosspartitionquery = true, maxdegreeofparallelism = -1, maxbuffereditemcount = -1 });          list<dynamic> transmittermmsilist = query.tolist(); //todo: remove duplicates         console.writeline("transmittermmsi count: {0}", transmittermmsilist.count());         hashset<string> exist = new hashset<string>();          foreach (var item in transmittermmsilist)         {             //int transmittermmsi = int32.parse(item.transmittermmsi.tostring());             string transmittermmsi = item.transmittermmsi.tostring();              if (exist.contains(transmittermmsi))             {                 continue;             }             exist.add(transmittermmsi);             console.writeline("transmittermmsi: {0} being processed.", transmittermmsi);             var response = await this.client.executestoredprocedureasync<string>(sproc.selflink,                 new requestoptions { partitionkey = new partitionkey(transmittermmsi) }, transmittermmsi, 30);             string s = response.response;             console.writeline("transmittermmsi: {0} processed completely.", transmittermmsi);         }     }      private async task trydeletestoredprocedure(string collectionselflink, string sprocid)     {         storedprocedure sproc = this.client.createstoredprocedurequery(collectionselflink).where(s => s.id == sprocid).asenumerable().firstordefault();         if (sproc != null)         {             await client.deletestoredprocedureasync(sproc.selflink);         }     } 

i tried comment 2 loops in js codes, documents.length output, while response number still less. however, changed query select a.id, documents.length correct. looks continuation issue.

the sproc timing out. use continuation token in these circumstances, need return c# calling code make call sproc passing in token. if show sproc code can more.


Comments

Popular posts from this blog

node.js - Node js - Trying to send POST request, but it is not loading javascript content -

javascript - Replicate keyboard event with html button -

javascript - Web audio api 5.1 surround example not working in firefox -