Create scala filter conditions based on JSON in API gateway -


i getting json filter conditions spark sql. json of format:

{     "x": {         "lt": "2"     } }

which should become like: spark.sql("select * df x < 2")

any idea how can proceed? data read parquet file using 

spark.read.parquet(filepath)

so code follows:

val df = spark.read.parquet(filepath) implicit val formats = org.json4s.defaultformats parse(filterjson).extract[map[string, any]] // once tables have been registered, can run sql queries on them. for((k,v)<-map){   v match {     case "eq"=>"==="     case "gt"=>">"     case "lt"=>"<"   } } df.filter(k)

any appreciated.


Comments

Post a Comment

Popular posts from this blog

html - How to custom Bootstrap grid height? -

why bootstrap grid has height 591.067 ? did height came ? .main-post { padding: 20px; background-color: #fff; border: 1px solid #d8d8d8; margin-bottom: 100px; } .main-post .post-categories { margin-bottom: 10px } .main-post h3 { margin: 0 0 10px; color: #777; letter-spacing: -1px; font-weight: bold } .main-post .post-author, .main-post .post-date, .main-post .post-comments { font-size: 12px } .main-post img { display: block; margin: 10px 0; } .main-post .post-summary { line-height: 1.7; color: #888 } .main-post { color: #999 } <div class="col-sm-6"> <div class="main-post"> <h3 class="post-title"> <a href="#"> post title </a> </...
Read more

javascript - pass values from mssql to views in node -

being new node js , javascript . have been trying find solution problem . this app.js file var express=require ('express'); var bodyparser = require('body-parser'); var path = require ('path'); var sql = require('mssql'); var app = express(); const config = { user: 'sa', password: 'password', server: 'localhost\\sqlexpress', // can use 'localhost\\instance' connect named instance database: 'pcgdb', options: { encrypt: false // use if you're on windows azure } }; const request = new sql.request() //view engine app.set('view engine', 'ejs'); app.set('views', path.join(__dirname, 'views')) app.use(bodyparser.json()); app.use(bodyparser.urlencoded({extended: false})); app.get('/',function(req,res){ res.render('index') }); app.get('/new_project', function(req, res) { function(data,err){ ...
Read more

javascript - Understanding ExtJS Framework Syntax -

i have been given project uses sencha's extjs framework, no handover or documentation. trying come grips hoping people can pls assist me following syntax possibly point me specific site might explain code newcomer framework. i have been going through code , unsure doing, i.e.: if(val==1) { ext.getcmp('status').settext('awaiting approv'); } switch(flag) { case 1: if(val==0) { ext.getcmp('complex_first').el.replacecls('x-form-complex', 'x-form-simplex'); } else { ext.getcmp('complex_first').el.replacecls('x-form-simplex', 'x-form-complex'); } break; case 2: if(val==0) { ext.getcmp('complex_second').el.replacecls('x-form-complex', 'x-form-simplex'); } else { ext.getcmp('complex_second').el.replacecls('x-fo...
Read more