← Back to Blog

How to Make Jupyter Notebooks Faster

Jupyter Notebooks are fantastic tools for coding, especially when dealing with data. But sometimes, they can run slowly, which can be frustrating. Let’s talk about a few ways you can make your notebooks run faster.
By Saturn Cloud | | Miscellaneous

Introduction

Jupyter Notebooks are fantastic tools for coding, especially when dealing with data. But sometimes, they can run slowly, which can be frustrating. Let’s talk about a few ways you can make your notebooks run faster.

Swap Loops for Vectorization

In Python, ‘loops’ (like ‘for’ and ‘while’) can be slow. ‘Vectorization’ is a technique where you perform an operation on a whole array of numbers at once. It’s usually faster than loops. So, try to use vectorization instead of loops where you can.

Use NumPy and Pandas

Both ‘NumPy’ and ‘Pandas’ are tools that can help your code run faster. NumPy helps with numerical calculations, and Pandas is excellent for dealing with data tables. They’re both designed to be quick, so use them where you can.

Don’t Display Too Much Data

It can be tempting to try and display all your data at once in a Jupyter notebook. But this can slow things down. Try to only display the data you need. You can do this by using commands like .head() or .sample() in Pandas, which show you only a small portion of your data.

Use %%time and %%timeit Magic Commands

In Jupyter, magic commands are special commands that start with a %. The %%time and %%timeit commands are really useful for figuring out what parts of your code are slow. %%time tells you how long a single cell takes to run, and %%timeit runs a cell several times and gives you the average time. You can use these commands to figure out what parts of your code you need to speed up.

Limit Output Logging

By default, Jupyter Notebooks display the output of each executed cell, including log messages. This logging can be resource-intensive, especially if a cell generates a large amount of output. To improve performance, you can reduce the amount of output logged or disable logging altogether.

Placing %%capture at the beginning of a cell captures the output, preventing it from being displayed. This is useful when you don’t need to see the output but still want to execute the cell.

Additionally, you can use the semicolons to suppress output. Appending a semicolon (;) at the end of a line in a code cell suppresses the output of that line. This is helpful when you want to avoid displaying the output of a specific line without capturing all output.

Last, you can disable logging in specific cells. By disabling logging for specific cells using the logging module. Set the logging level to a higher value or disable it altogether using logging.disable() to prevent log messages from being generated and displayed.

Clear Output

If you have already executed a cell and don’t need its output anymore, you can clear the output to reduce clutter and improve notebook performance. Use the “Clear Output” option in the Jupyter Notebook toolbar or execute Cell -> All Output -> Clear from the menu.

Alt text

Use Smaller Data When Testing

When you’re testing your code, you don’t need to run it on all your data. Try running it on a small sample of your data instead. This can make your testing a lot quicker.

Common Errors and How to Handle Them

Memory Issues

Large datasets may lead to memory errors. Consider using techniques like chunking or optimizing data types to reduce memory usage.

Kernel Crashes

Kernel crashes can occur due to long-running processes. Use %xmode magic command to debug crashes and identify problematic code sections.

Upgrade Your Hardware or Use Cloud-Based Solutions

If you’re dealing with really big data, you might need more powerful hardware. Consider upgrading your computer’s RAM or CPU. Alternatively, you can use free cloud-based solutions like Saturn Cloud, which lets you run your notebooks on powerful servers in the cloud.

Remember, every notebook and every piece of code is different. These tips might not all apply to your specific situation, but they’re a good starting point.

You may also be interested in:

About Saturn Cloud

Saturn Cloud is your all-in-one solution for data science & ML development, deployment, and data pipelines in the cloud. Spin up a notebook with 4TB of RAM, add a GPU, connect to a distributed cluster of workers, and more. Request a demo today to learn more.

Get a Technical Demo