These days there is a tendency in data analysis to use Jupyter Notebooks. But what happens if you have too many jupyter notebooks? For example, there are more than a hundred.<p>Actually, you start creating some modules. However, it is less convenient to work with them compared to what was before. It happens that you should code in web interface, somewhere in similar to the notepad++ form or you should change your IDLE.<p>Personally, I work in Pycharm and so far I couldn't assess remote interpreter or VCS. It is because pickle files or word2vec weighs too much (3gb+) and so I don't want to download/upload them.
Also Jupyter is't cool in pycharm.<p>Do you have better practices in your companies?
How to correctly adjust IDLE?
Do you know about any possible substitution for the IPython notebook in the world of data analysis?
> what happens if you have too many jupyter notebooks? For example, there are more than a hundred.<p>Like anything else, Jupyter Notebook is limited by the CPU and RAM of the system hosting the Tornado server and Jupyter kernels.<p>At 100 notebooks (or even just one), it may be a good time to factor common routines into a packaged module with tests and documentation.<p>It's actually possible (though inefficient) to import code from Jupyter notebooks with ipython/ipynb (pypi:ipynb):
<a href="https://github.com/ipython/ipynb" rel="nofollow">https://github.com/ipython/ipynb</a>
( <a href="https://jupyter-notebook.readthedocs.io/en/stable/examples/Notebook/Importing%20Notebooks.html" rel="nofollow">https://jupyter-notebook.readthedocs.io/en/stable/examples/N...</a> )<p>> Actually, you start creating some modules. However, it is less convenient to work with them compared to what was before. It happens that you should code in web interface, somewhere in similar to the notepad++ form or you should change your IDLE.<p>The Spyder IDE has support for .ipynb notebooks converted to .py (which have the IPython prompt markers in them). Spyder can connect an interpreter prompt to a running IPython/Jupyter kennel. There's also a Spyder plugin for Jupyter Notebook:
<a href="https://github.com/spyder-ide/spyder-notebook" rel="nofollow">https://github.com/spyder-ide/spyder-notebook</a><p>> Personally, I work in Pycharm and so far I couldn't assess remote interpreter or VCS. It is because pickle files or word2vec weighs too much (3gb+) and so I don't want to download/upload them.<p>Remote data access times can be made faster by increasing the space efficiency of the storage format, increasing the bandwidth of the connection, moving the data to the code, or moving the code to the data.<p>> Do you have better practices in your companies?<p>There are a number of [Reproducible] Data Science cookiecutter templates which have a directory for notebooks, module packaging, and Sphinx docs: <a href="https://cookiecutter.readthedocs.io/en/latest/readme.html#data-science" rel="nofollow">https://cookiecutter.readthedocs.io/en/latest/readme.html#da...</a><p>Refactoring increases testability and code reuse.<p>> How to correctly adjust IDLE?<p>I don't think I understand the question?<p>"Configuring IPython" <a href="https://ipython.readthedocs.io/en/stable/config/index.html" rel="nofollow">https://ipython.readthedocs.io/en/stable/config/index.html</a><p>Jupyter > "Installation, Configuration, and Usage" <a href="https://jupyter.readthedocs.io/en/latest/projects/content-projects.html#" rel="nofollow">https://jupyter.readthedocs.io/en/latest/projects/content-pr...</a><p>> Do you know about any possible substitution for the IPython notebook in the world of data analysis?<p>From <a href="https://en.wikipedia.org/wiki/Notebook_interface" rel="nofollow">https://en.wikipedia.org/wiki/Notebook_interface</a> :<p>> > "Examples of the notebook interface include the Mathematica notebook, Maple worksheet, MATLAB notebook, IPython/Jupyter, R Markdown, Apache Zeppelin, Apache Spark Notebook, and the Databricks cloud."<p>There are lots of Jupyter kernels for different tools and languages (over 100; including for other 'notebook interfaces'):
<a href="https://github.com/jupyter/jupyter/wiki/Jupyter-kernels" rel="nofollow">https://github.com/jupyter/jupyter/wiki/Jupyter-kernels</a><p>And there are lots of Jupyter integrations and extensions: <a href="https://github.com/quobit/awesome-python-in-education/blob/master/README.md#jupyter" rel="nofollow">https://github.com/quobit/awesome-python-in-education/blob/m...</a>