Datanode 进程未在 Hadoop 中运行

2024-02-26

我使用以下命令设置并配置了多节点 Hadoop 集群本教程 http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/#formatting-the-hdfs-filesystem-via-the-namenode.

当我输入 start-all.sh 命令时,它显示所有进程都正确初始化,如下所示:

starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-namenode-jawwadtest1.out
jawwadtest1: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-datanode-jawwadtest1.out
jawwadtest2: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-datanode-jawwadtest2.out
jawwadtest1: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-secondarynamenode-jawwadtest1.out
starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-jobtracker-jawwadtest1.out
jawwadtest1: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-tasktracker-jawwadtest1.out
jawwadtest2: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-root-tasktracker-jawwadtest2.out

但是,当我输入 jps 命令时,我得到以下输出:

31057 NameNode
4001 RunJar
6182 RunJar
31328 SecondaryNameNode
31411 JobTracker
32119 Jps
31560 TaskTracker

如您所见,没有 datanode 进程在运行。我尝试配置单节点集群,但遇到了同样的问题。有人知道这里可能出了什么问题吗?是否有教程中未提及或我可能已经查看过的配置文件?我是 Hadoop 新手,有点迷失,任何帮助将不胜感激。

编辑: hadoop-root-datanode-jawwadtest1.log:

STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.3
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/$
************************************************************/
2012-08-09 23:07:30,717 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loa$
2012-08-09 23:07:30,734 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapt$
2012-08-09 23:07:30,735 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:$
2012-08-09 23:07:30,736 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl:$
2012-08-09 23:07:31,018 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapt$
2012-08-09 23:07:31,024 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl:$
2012-08-09 23:07:32,366 INFO org.apache.hadoop.ipc.Client: Retrying connect to $
2012-08-09 23:07:37,949 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: $
        at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(Data$
        at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransition$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNo$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNod$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode($
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataN$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.$
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1$

2012-08-09 23:07:37,951 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: S$
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at jawwadtest1/198.101.220.90
************************************************************/

你需要做这样的事情:

  • bin/stop-all.sh (or stop-dfs.sh and stop-yarn.sh在 2.x 系列中)
  • rm -Rf /app/tmp/hadoop-your-username/*
  • bin/hadoop namenode -format (or hdfs在 2.x 系列中)

解决方案取自:http://pages.cs.brandeis.edu/~cs147a/lab/hadoop-troubleshooting/ http://pages.cs.brandeis.edu/~cs147a/lab/hadoop-troubleshooting/。基本上它包括从头开始重新启动,因此请确保格式化 hdfs 不会丢失数据。

本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)

Datanode 进程未在 Hadoop 中运行 的相关文章

随机推荐