我正在努力将 Spark 添加到我的 python 路径中:
(myenv)me@me /home/me$ set SPARK_HOME="/home/me/spark-1.2.1-bin-hadoop2.4"
(myenv)me@me /home/me$ set PYTHONPATH=$PYTHONPATH:$SPARK_HOME:$SPARK_HOME/python:$SPARK_HOME/python/build:$SPARK_HOME/bin
(myenv)me@me /home/me$ python -c 'import sys; print(sys.path)'
['', '/home/me/.virtualenvs/default/lib/python2.7', '/home/me/.virtualenvs/default/lib/python2.7/plat-x86_64-linux-gnu', '/home/me/.virtualenvs/default/lib/python2.7/lib-tk', '/home/me/.virtualenvs/default/lib/python2.7/lib-old', '/home/me/.virtualenvs/default/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/home/me/.virtualenvs/default/local/lib/python2.7/site-packages', '/home/me/.virtualenvs/default/lib/python2.7/site-packages']
(myenv)me@me /home/me$ python -c 'import pyspark'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named pyspark
我也有同样的问题,但是this https://stackoverflow.com/questions/23256536/importing-pyspark-in-python-shell?lq=1 helped.
只需在 .bashrc 中添加以下命令
export SPARK_HOME=/path/to/your/spark-1.4.1-bin-hadoop2.6
export PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/build:$PYTHONPATH
export PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.8.2.1-src.zip:$PYTHONPATH
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)