hadoop - Perform sql join using sqlcontext spark -
i tried running query querying oracle db joins using sqlcontext like,
val sql="select b,c b.join=c.join" val dataframe = sqlcontext.read.jdbc(url,sql,connection_properties) i getting invalid tablename error. if try querying table below works fine.
val df1 = sqlcontext.read.jdbc(url,"b",connection_properties) val df2 = sqlcontext.read.jdbc(url,"c",connection_properties) will not possible run join queries using sqlcontext.
this need do, create 2 dataframes tables below
val df1 = sqlcontext.read.jdbc(url,"b",connection_properties) val df2 = sqlcontext.read.jdbc(url,"c",connection_properties) and join 2 dataframes key want join
df1.join(df2, <join condition>, <which join>) //example df1.join(df2, df1($"id") === df2($"id"), "left") i think better option far know
hope helps!
Comments
Post a Comment