hadoop - Errors when using Sqoop action in Oozie editor (Hue) -
i trying use sqoop action in oozie editor in hue, can't work.
here's have tried far.
i put in arguments, instead of command (http://alvincjin.blogspot.com.au/2014/06/create-sqoop-action-in-oozie-using-hue.html)
further, trying connect teradata, i've placed jdbc jars in hdfs , have added them in files. current workflow looks in editor: sqoop action .
the workflow definition is:
<workflow-app name="sqoop_test" xmlns="uri:oozie:workflow:0.5"> <start to="sqoop-b20d"/> <kill name="kill"> <message>action failed, error message[${wf:errormessage(wf:lasterrornode())}]</message> </kill> <action name="sqoop-b20d"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobtracker}</job-tracker> <name-node>${namenode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>development</value> </property> <property> <name>mapred.job.name</name> <value>test_sqoop</value> </property> <property> <name>mapred.task.timeout</name> <value>0</value> </property> </configuration> <arg>import</arg> <arg>--connect</arg> <arg>jdbc:teradata://xxxxx</arg> <arg>--query</arg> <arg>select count(*) xxxxx</arg> <arg>--fetch-size</arg> <arg>10000</arg> <arg>--num-mappers</arg> <arg>1</arg> <arg>--hive-table-name</arg> <arg>xxxxx.tmp_sqoop_test</arg> <arg>--hive-import</arg> <arg>--hive-overwrite</arg> <arg>--target-dir</arg> <arg>/user/dxxxxx/digital/test/tmp_sqoop_test</arg> <arg>--username</arg> <arg>xxxxx</arg> <arg>--password</arg> <arg>xxxxx</arg> <file>/user/hue/oozie/workspaces/digital/lib/terajdbc4.jar#terajdbc4.jar</file> <file>/user/hue/oozie/workspaces/digital/lib/teradata-connector-1.3.4-hadoop220.jar#teradata-connector-1.3.4-hadoop220.jar</file> </sqoop> <ok to="end"/> <error to="kill"/> </action> <end name="end"/> </workflow-app> however, error:
2016-01-06 14:13:52,114 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(296)) - error parsing arguments import: 2786 [main] error org.apache.sqoop.tool.basesqooptool - unrecognized argument: --hive-table-name 2016-01-06 14:13:52,114 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(299)) - unrecognized argument: --hive-table-name 2786 [main] error org.apache.sqoop.tool.basesqooptool - unrecognized argument: xxxxx.tmp_sqoop_test 2016-01-06 14:13:52,114 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(299)) - unrecognized argument: tdcprdr_app_digital.tmp_sqoop_test 2786 [main] error org.apache.sqoop.tool.basesqooptool - unrecognized argument: --hive-import 2016-01-06 14:13:52,114 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(299)) - unrecognized argument: --hive-import 2786 [main] error org.apache.sqoop.tool.basesqooptool - unrecognized argument: --hive-overwrite 2016-01-06 14:13:52,114 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(299)) - unrecognized argument: --hive-overwrite 2787 [main] error org.apache.sqoop.tool.basesqooptool - unrecognized argument: --target-dir 2016-01-06 14:13:52,115 error [main] tool.basesqooptool (basesqooptool.java:hasunrecognizedargs(299)) - unrecognized argument: --target-dir ... i under impression error can resolved placing in arguments. same code works when run through shell script. i've tried placing import command , connection string in command section, doesn't run. i've tried creating minimalistic sqoop action well, query , connect statement follows:
<workflow-app name="sqoop_minimal" xmlns="uri:oozie:workflow:0.5"> <start to="sqoop-eeeb"/> <kill name="kill"> <message>action failed, error message[${wf:errormessage(wf:lasterrornode())}]</message> </kill> <action name="sqoop-eeeb"> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobtracker}</job-tracker> <name-node>${namenode}</name-node> <arg>import</arg> <arg>--connect</arg> <arg>jdbc:teradata://tdxxxxx</arg> <arg>--query</arg> <arg>select count(*) xxxxx</arg> <arg>--target-dir</arg> <arg>/user/dxxxxx/digital/test/tmp_sqoop_test</arg> <arg>--username</arg> <arg>xxxxx</arg> <arg>--password</arg> <arg>xxxxx</arg> <file>/user/hue/oozie/workspaces/digital/lib/teradata-connector-1.3.4-hadoop220.jar#teradata-connector-1.3.4-hadoop220.jar</file> <file>/user/hue/oozie/workspaces/digital/lib/terajdbc4.jar#terajdbc4.jar</file> </sqoop> <ok to="end"/> <error to="kill"/> </action> <end name="end"/> </workflow-app> with workflow, vague error follows:
>>> invoking sqoop command line >>> 2287 [main] warn org.apache.sqoop.tool.sqooptool - $sqoop_conf_dir has not been set in environment. cannot check additional configuration. 2016-01-06 14:57:48,381 warn [main] tool.sqooptool (sqooptool.java:loadpluginsfromconfdir(175)) - $sqoop_conf_dir has not been set in environment. cannot check additional configuration. 2324 [main] info org.apache.sqoop.sqoop - running sqoop version: 1.4.5.3.0.0.0-249 2016-01-06 14:57:48,418 info [main] sqoop.sqoop (sqoop.java:<init>(92)) - running sqoop version: 1.4.5.3.0.0.0-249 2339 [main] warn org.apache.sqoop.tool.basesqooptool - setting password on command-line insecure. consider using -p instead. 2016-01-06 14:57:48,433 warn [main] tool.basesqooptool (basesqooptool.java:applycredentialsoptions(1014)) - setting password on command-line insecure. consider using -p instead. intercepting system.exit(1) <<< invocation of main class completed <<< failing oozie launcher, main class [org.apache.oozie.action.hadoop.sqoopmain], exit code [1] oozie launcher failed, finishing hadoop job gracefully the oozie version 4.1.0.3.0.0.0-249.
i've tried searching solution online, no luck. appreciated. thank you!
already seen , tried links:
https://community.cloudera.com/t5/batch-processing-and-workflow/sqoop-fails-with-quot-error-parsing-arguments-for-import-quot/td-p/31930 http://stackoverflow.com/questions/25770698/sqoop-free-form-query-causing-unrecognized-arguments-in-hue-oozie
there no such argumnets sqoop --hive-table-name use --hive-table. should not show unrecognized argument now
Comments
Post a Comment