当我设置HADOOP_HOME=/cygdrive/c/ecosystem/hadoop-2.5.1然后尝试运行bin/hadoop 文件系统 or bin/hadoop hadoop-streaming.jar从 Cygwin,我收到以下错误:
ERROR [main] util.Shell (Shell.java:getWinUtilsPath(373)) - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:432)
at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:478)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
Exception in thread "main" java.lang.RuntimeException: core-site.xml not found
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2269)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2112)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:989)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:961)
at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1299)
at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:319)
at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:479)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:340)
我可以访问bin/hadoop 版本随着类路径和大多数其他命令。我想也许我需要一个 Windows 风格的路径,以便它可以符合 \bin\winutils 风格。所以,我设置HADOOP_HOME=c:\ecosystem\hadoop-2.5.1这个错误就消失了。但是,现在我在 Cygwin 中运行时收到链接错误。我测试了hadoopHADOOP_HOME=c:\ecosystem\hadoop-2.5.1通过 Windows 命令,我可以毫无问题地运行 MapReduce 作业。
我想在Cygwin中成功运行hadoop,并且不想处理链接错误。如果我保留HADOOP_HOME=/cygdrive/c/ecosystem/hadoop-2.5.1,如果 Cygwin 路径失败或为空,有什么方法可以设置 hadoop-config 或其他文件来查找 Windows 样式路径? hadoop-config 中可能有这样的内容:
if ["$HADOOP_HOME" = ""]; then
HADOOP_HOME=$(cygpath -w "$HADOOP_HOME")
fi
显然这个特定的代码在 hadoop-config 中不起作用,但我想你会明白的。任何帮助,将不胜感激。
当我尝试在Windows中本地执行Hadoop MR程序时,通过Eclipse设置环境变量HADOOP_HOME:运行->运行配置->环境,我用Eclipse解决了类似的错误。
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)