Fehler beim Laufen Sqoop Import - ERROR manager.OracleManager: Fehler Transaktion rückgängig zu machen

Question

ich Sqoop bin mit db zum Laden von Daten aus Oracle

Spezifikation hive:

CDH-5.5.1 Sqoop 1.4 .2

Initialisation:

export SQOOP_HOME=/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/sqoop 
    export HIVE_HOME=/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hive 
    export HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/sqoop/lib/ojdbc7.jar:/opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/lib/hive/lib/* 
    export HADOOP_USER_NAME=hdfs 
    export PATH=$PATH:$HIVE_HOME/bin 

Sqoop Befehl:

sqoop import --connect jdbc:oracle:thin:@<host>/<pluggable DB SID> --username username --password password --table tablename -m 1 --hive-import --hive-table hivetablename 

Bei der Ausführung, erhalte ich die folgenden Fehler

16/06/01 16:11:37 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM VEHICLEDETAILS t WHERE 1=0 
    16/06/01 16:11:37 WARN manager.SqlManager: SQLException closing ResultSet: java.sql.SQLException: Could not commit with auto-commit set on 
    16/06/01 16:11:37 ERROR manager.OracleManager: Failed to rollback transaction 
    java.sql.SQLException: Could not rollback with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2423) 
     at org.apache.sqoop.manager.OracleManager.getColumnNames(OracleManager.java:767) 
     at org.apache.sqoop.orm.ClassWriter.getColumnNames(ClassWriter.java:1207) 
     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1062) 
     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:37 ERROR manager.OracleManager: Failed to list columns 
    java.sql.SQLException: Could not commit with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:2356) 
     at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:2403) 
     at org.apache.sqoop.manager.OracleManager.getColumnNames(OracleManager.java:764) 
     at org.apache.sqoop.orm.ClassWriter.getColumnNames(ClassWriter.java:1207) 
     at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1062) 
     at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:37 INFO orm.CompilationManager: HADOOP_HOME is /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p0.11/bin/../lib/sqoop/../hadoop 
    Note: /tmp/sqoop-root/compile/1d122ab0d74da045fd45da7a765f595d/VEHICLEDETAILS.java uses or overrides a deprecated API. 
    Note: Recompile with -Xlint:deprecation for details. 
    16/06/01 16:11:38 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/1d122ab0d74da045fd45da7a765f595d/VEHICLEDETAILS.jar 
    16/06/01 16:11:38 ERROR manager.OracleManager: Failed to rollback transaction 
    java.lang.NullPointerException 
     at org.apache.sqoop.manager.OracleManager.getPrimaryKey(OracleManager.java:828) 
     at org.apache.sqoop.manager.SqlManager.getSplitColumn(SqlManager.java:408) 
     at org.apache.sqoop.manager.SqlManager.checkTableImportOptions(SqlManager.java:427) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:462) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:38 ERROR manager.OracleManager: Failed to list columns 
    java.sql.SQLException: Could not rollback with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2423) 
     at org.apache.sqoop.manager.OracleManager$ConnCache.getConnection(OracleManager.java:190) 
     at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:283) 
     at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) 
     at org.apache.sqoop.manager.OracleManager.getPrimaryKey(OracleManager.java:814) 
     at org.apache.sqoop.manager.SqlManager.getSplitColumn(SqlManager.java:408) 
     at org.apache.sqoop.manager.SqlManager.checkTableImportOptions(SqlManager.java:427) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:462) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:39 INFO manager.OracleManager: Time zone has been set to GMT 
    16/06/01 16:11:39 ERROR manager.OracleManager: Failed to rollback transaction 
    java.sql.SQLException: Could not rollback with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2423) 
     at org.apache.sqoop.manager.OracleManager.getPrimaryKey(OracleManager.java:828) 
     at org.apache.sqoop.manager.SqlManager.getSplitColumn(SqlManager.java:408) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:464) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:39 ERROR manager.OracleManager: Failed to list columns 
    java.sql.SQLException: Could not commit with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:2356) 
     at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:2403) 
     at org.apache.sqoop.manager.OracleManager.getPrimaryKey(OracleManager.java:825) 
     at org.apache.sqoop.manager.SqlManager.getSplitColumn(SqlManager.java:408) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:464) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:39 INFO mapreduce.ImportJobBase: Beginning import of VEHICLEDETAILS 
    16/06/01 16:11:39 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 
    16/06/01 16:11:39 ERROR manager.OracleManager: Failed to rollback transaction 
    java.lang.NullPointerException 
     at org.apache.sqoop.manager.OracleManager.getColumnNames(OracleManager.java:767) 
     at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureInputFormat(DataDrivenImportJob.java:164) 
     at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:194) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:39 ERROR manager.OracleManager: Failed to list columns 
    java.sql.SQLException: Could not rollback with auto-commit set on 
     at oracle.jdbc.driver.PhysicalConnection.rollback(PhysicalConnection.java:2423) 
     at org.apache.sqoop.manager.OracleManager$ConnCache.getConnection(OracleManager.java:190) 
     at org.apache.sqoop.manager.OracleManager.makeConnection(OracleManager.java:283) 
     at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52) 
     at org.apache.sqoop.manager.OracleManager.getColumnNames(OracleManager.java:751) 
     at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureInputFormat(DataDrivenImportJob.java:164) 
     at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:194) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 
    16/06/01 16:11:39 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 
    16/06/01 16:11:39 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 
    16/06/01 16:11:39 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 
    16/06/01 16:11:40 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/tmp/hadoop-root/mapred/staging/hdfs1761993336/.staging/job_local1761993336_0001 
    Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected 
     at org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:53) 
     at com.cloudera.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36) 
     at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:121) 
     at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:304) 
     at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:321) 
     at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:199) 
     at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307) 
     at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:422) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) 
     at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304) 
     at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1325) 
     at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:141) 
     at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:202) 
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:465) 
     at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:380) 
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:403) 
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476) 
     at org.apache.sqoop.Sqoop.run(Sqoop.java:145) 
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220) 
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229) 
     at org.apache.sqoop.Sqoop.main(Sqoop.java:238) 

I ojdbc 7 Glas und Oracle 12c Datenbank und jdk 8

Answer 1

Ich benutze die folgende Abfrage, um Daten in Hive-Tabelle von MySQL zu importieren. Es sollte auch mit Oracle funktionieren. Nur müssen Sie die JDBC-URL ändern

sqoop import --connect jdbc:mysql://localhost:3306/retail_db --username retail_dba -P --table customers --target-dir /temp1 --hive-table stackOverflow --hive-import 

INFORMIEREN SIE MICH BITTE, wenn es hilft

Answer 2

Verwenden "--query select * from VEHICLEDETAILS" instead of "--table VEHICLEDETAILS"

bin mit

Diesen Beitrag anzeigen https://discuss.zendesk.com/hc/en-us/articles/204206528-Sqoop-import-fails-to-import-oracle-table-with-Could-not-commit-with-auto-commit-set-on-