java - Spark submit: Table or view not found using jar -


when run hiveread.java intellij ide can run , result. created jar file ( it's maven project ) , tried run ide, gave me 

classloaderresolver class "" gave error on creation : {1}

then looked @ answers , found had add datanulcues jars, did this

java -jar /home/saurab/sparkprojects/spark_hive/target/myjar-jar-with-dependencies.jar --jars jars/datanucleus-api-jdo-3.2.6.jar,jars/datanucleus-core-3.2.10.jar,jars/datanucleus-rdbms-3.2.9.jar,/home/saurab/hadoopec/hive/lib/mysql-connector-java-5.1.38.jar

then got error

org.datanucleus.exceptions.nucleususerexception: persistence process has been specified use classloaderresolver of name "datanucleus" yet has not been found datanucleus plugin mechanism. please check classpath , plugin specification.

somewhere found should spark-submit. did this

./bin/spark-submit --class hiveread --master yarn  --jars jars/datanucleus-api-jdo-3.2.6.jar,jars/datanucleus-core-3.2.10.jar,jars/datanucleus-rdbms-3.2.9.jar,/home/saurab/hadoopec/hive/lib/mysql-connector-java-5.1.38.jar --files /home/saurab/hadoopec/spark/conf/hive-site.xml /home/saurab/sparkprojects/spark_hive/target/myjar-jar-with-dependencies.jar

now new type of error

table or view not found: `bigmart`.`o_sales`;

help me !! :)

i have copied hive-site.xml /spark/conf, started hive-metastore service ( hiveserver2 --service metastore )

here hiveread.java code if interested.


Comments

Post a Comment

Popular posts from this blog

python - Selenium remoteWebDriver (& SauceLabs) Firefox moseMoveTo action exception -

while performing simple hover test on firefox through selenium (3.4.0) python bindings (3.4.3) using of available os in saucelabs (except linux, latest available firefox version old), performing following actionchain: hover = actionchains(driver) hover.move_to_element(elm_men_menu).perform() it throws following error: taceback: file [...] in testhover hover.perform() file "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/common/action_chains.py", line 80, in perform action() file "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/common/action_chains.py", line 290, in <lambda> command.move_to, {'element': to_element.id})) file "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute self.error_handler.check_response(response) file "/usr/local/lib/python2.7/...
Read more

html - How to custom Bootstrap grid height? -

why bootstrap grid has height 591.067 ? did height came ? .main-post { padding: 20px; background-color: #fff; border: 1px solid #d8d8d8; margin-bottom: 100px; } .main-post .post-categories { margin-bottom: 10px } .main-post h3 { margin: 0 0 10px; color: #777; letter-spacing: -1px; font-weight: bold } .main-post .post-author, .main-post .post-date, .main-post .post-comments { font-size: 12px } .main-post img { display: block; margin: 10px 0; } .main-post .post-summary { line-height: 1.7; color: #888 } .main-post { color: #999 } <div class="col-sm-6"> <div class="main-post"> <h3 class="post-title"> <a href="#"> post title </a> </...
Read more

Ansible warning on jinja2 braces on when -

i using code in vars my_var: "{{lookup('env','my_var') | default(true, true)}}" i using like - include: task.yml when: my_var this worked without issue in ansible 2.2 in ansible 2.3 warning this [warning]: when statements should not include jinja2 templating delimiters such {{ }} or {% %}. found: {{lookup('env','my_var') | default(true, true)}} so if use my_var: "lookup('env','my_var') | default(true, true)" , mean remove {{ }} , no warning my_var evaluated true no matter if pass false env variable. works fine in previous code gave how can fix it? the cleanest way set fact instead of using template in variable (it prevent lookup plugin being called multiple times in included tasks - not huge performance gain, cleaner): - set_fact: my_var: "{{lookup('env','my_var') | default(true, true)}}" also mind evaluate true in cases when environment my_var diff...
Read more