ImportError: No module named pandas - PySpark

Getting an error “ImportError: No module named pandas”. Can you please install Pandas soon?

thank you!

Something unexpected may happened, pandas is already installed in the Lab.
Once can you restart your server or login again and import it and let me know ?

send the screenshots if still getting the same.

Thank you for the reply. Appreciated.
Attached are the screenshots. I am still getting the error while trying to import pandas for spark.

Interesting.

There are two possibility.

  1. Either Pyspark pandas need to be installed using “pip install pyspark-pandas” and is different from normal pandas.
    refer below.
    https://pypi.org/project/pyspark-pandas/

Or
2) pandas need to be installed in the server side spark use pip list|grep ‘pandas’ to find it.
from the screenshots and I also done I am getting the same.

All the best!

Yes, it needs to be installed. I tried to install, but failed with “OSError: [Errno 13] Permission denied: ‘/usr/bin/pip’”. Attached is the screenshot.

Would like to request Admin to install it.

Thank you!

I figured it out!

This post helped me. Thank you!

Yes, you are right. But still I would like to request “Lab Admin” to install it on server side. So that I do not have to activate/deactivate VRV every time I login and also I need many other modules form server side.