About Notebook Sessions

Learn how to work with your Data Science notebook sessions, which are interactive coding environments for building and training models.

After you choose a project, the Project Details page is displayed with a list of notebook sessions. This page is used to:

  • Create notebook sessions.

  • Select a notebook session to view its details and work with it.

  • Use the Actions menu to view details, edit, move a notebook resource, or delete a notebook session.

  • OCID: Determine the OCID of a resource. A shortened version of the OCID is displayed though you can use Show and Hide to toggle the display of the OCID. Use the Copy link to save the entire OCID to your clipboard to paste it elsewhere. For example, you could paste into a file and save it, then use it in your model scripts.

  • Use the List Scope filter to view notebook sessions associated with your selected project in another compartment.

  • Filter notebook sessions by status using the Notebook Session State drop-down list. The default is to view all status types.

  • When there are tags applied to notebook sessions, you can further filter notebook sessions by clicking add or clear next to Tag Filters.

  • Select other Data Science resources, like models. model deployments, and jobs.

Creating Notebook Sessions

You create a notebook session to access a JupyterLab interface using a customizable Compute, storage, and network configuration.

  1. Log into your tenancy using the Console with the necessary policies.
  2. Open the navigation menu and click Analytics & AI. Under Machine Learning, click Data Science.
  3. Select a compartment.
  4. Click the name of the project to contain the notebook session.
  5. Click Create notebook session.
  6. Select the compartment that you want to contain the notebook session.
  7. (Optional) Enter a unique name for the notebook session (limit of 255 characters). If you don't provide a name, a name is automatically generated for you. For example, datasciencenotebooksession20200108222435.
  8. Select a Compute shape by clicking Select.
    1. Select one of the supported shape series.
    2. Select the shape that best suits how you want to use the notebook. For the AMD shape, you can use the default or set the number of OCPUs and memory.
    3. Click Submit.
    Tip

    Sizing Notebook Sessions gives you tips about how to use shapes.
  9. Enter the Block Volume size you want to use between 50 GB and 10,240 GB (10 TB). You can change the value by 1 GB increments. The default value is 1,024 GB. Persisting Data and Files on your Notebook Session Block Volume provides more details.
  10. Select the VCN compartment that contains the VCN that you want to use. You can change the compartment if necessary by clicking Change Compartment, and then selecting the new compartment that has the VCN you want to use.
  11. Select the subnet that you want to use. You can change the compartment if necessary by clicking Change Compartment, and then selecting the new compartment that has the subnet you want to use.
  12. (Optional) Add tags to the notebook session by selecting a tag namespace, then entering the key and the value. You can add more tags to the compartment by clicking +Additional Tags, see Working with Resource Tags.
  13. (Optional) View the details for your notebook session immediately after creation by selecting Display notebook session details after creation.
  14. Click Create.
You are advanced to the notebook sessions page. When it is complete, the status turns to active, and you can open the notebook session. The create notebook session action is asynchronous and initiates a work request. You can use the work request to track the status of an operation. For general information about using work requests in OCI, see Work Requests and the Work Requests API.

Opening Notebook Sessions

You can either click the notebook session's name or the Actions menu, and click Open to open the notebook session’s JupyterLab interface in another tab. For more information about using JupyterLab, see the JupyterLab Documentation.

Note

You can only open an active notebook session.

When you open a new notebook, the Launcher tab is open and contains the main actions. You can choose to:

Environment Explorer

Work with the various conda environments.

Notebook Examples

Learn how to use notebooks by opening a getting started or other task specific examples for installed conda environments.

If this button isn't displayed, then click the File menu, select New, and then select Notebook Examples to choose a notebook.

Each example is loaded with a temporary file name that you can change by right-clicking the tab name, and then clicking Rename Notebook.... Enter a new name, then click Rename.

Python and Installed Conda Environments

Python and any installed conda environments are listed. You can create a notebook session or start a kernel console.

The Python 3 kernel is the conda environment contains a minimal set of libraries, including oci, cx_oracle, and numpy. We recommend that you install or create more conda environments.

A new notebook opens and the first cell contains useful tips that are the same as when you create a notebook.

Other

You can open a terminal, create text or markdown files, or get help.

You can open a new Launcher tab by clicking the plus sign (+) in the File Browser.

Following are some additional tips for notebook sessions:

Git Extension

Get direct access to your Git repository using the Git extension included in notebook sessions. If you see the Failed to load the jupyterlab-git server extension warning, that means you are using an older version of Notebook Virtual Machine so the Git extension isn’t installed. You can click Dismiss to work in the notebook without issues. If you want to start using the Git extension, simply deactivate and reactivate the notebook to get the extension.

Public Internet Access

How to check for public internet access:

import requests 
response=requests.get("https://oracle.com") 
assert response.status_code==200, "Internet connection failed"
Authentication

To set up the OCI configuration file and API key follow the instructions in the api_keys.ipynb notebook. Access it in the Launcher tab by clicking Notebook Examples, selecting the file, and then click Load Example.

Cell Tips

Typical cell imports and settings:

%load_ext autoreload
%autoreload 2
%matplotlib inline

import warnings
warnings.filterwarnings('ignore')

import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.ERROR)

import ads
from ads.dataset.factory import DatasetFactory
from ads.automl.provider import OracleAutoMLProvider
from ads.automl.driver import AutoML
from ads.evaluations.evaluator import ADSEvaluator
from ads.common.data import MLData
from ads.explanations.explainer import ADSExplainer
from ads.explanations.mlx_global_explainer import MLXGlobalExplainer
from ads.explanations.mlx_local_explainer import MLXLocalExplainer
from ads.catalog.model import ModelCatalog
from ads.common.model_artifact import ModelArtifact
Environment Variables

Useful environment variables

import os

print(os.environ["NB_SESSION_COMPARTMENT_OCID"])
print(os.environ["PROJECT_OCID"])
print(os.environ["USER_OCID"])
print(os.environ["TENANCY_OCID"])
print(os.environ["NB_REGION"])

Viewing Notebook Session Details

To open a notebook session's detail page, you can either click the notebook session's name or the Actions menu, and select View Details.

From the Notebook Session Information tab, you can view the:

OCID

A shortened version of the OCID is displayed. You can use the Show and Hide links to toggle the display of the full OCID. Use the Copy link to save the entire OCID to your clipboard to paste it elsewhere. For example, you could paste into a file and save it, then use it in your model scripts.

Other Details

These details are displayed:

  • Created On: The date and time when the notebook session was created.

  • Created By: The email address of the user that created the notebook.

  • Block Storage Size (in GB): The defined Block Storage.

  • Compute Instance Shape: The defined Compute instance shape.

  • When applicable, the number of OCPUs and amount of memory that were defined.

VCN

The subnet that was selected is displayed as a link so that you can go to the VCN Details page to manage it.

Subnet

The subnet that was selected is displayed as a link so that you can go to the Subnet Details page to manage it.

Metrics
If metrics are configured, then they are displayed.

From the Tags tab, you can view the tags that have been applied to the model. To update or remove a tag, find the tag you want and click the pencil button next to it. Enter the new tag then save it or remove the tag by clicking Remove Tag.

Editing Notebook Sessions

You can either click the notebook session's name or the Actions menu, and click Edit. You can change the name of the notebook session, and then save your changes.

When a notebook session is active, only the name can be edited. To edit other options, you have to deactivate it. Then activate it, which allows you to change any of the options like the block storage size, Compute shape, compartment, VCN, and the subnet.

Applying Tags to Notebook Sessions

You can either click the notebook session's name or the Actions menu, and click Add Tags or the Tags tab.

You can apply defined and free form tags to notebook sessions, see Working with Resource Tags.

Persisting Data and Files on your Notebook Session Block Volume

Files and data on the local disk of a notebook session Compute are not persisted when the notebook session is deactivated or crashes. To persist files and data, you must save them to Oracle Cloud Infrastructure Block Volumes. A Block Volume is mounted in /home/datascience. You can access it from the JupyterLab file tree.

When you deactivate a notebook session, the files stored on the Block Volume are mounted back in /home/datascience after you activate your notebook session. To back up your data, push your code changes to a remote Git repository and save the data to OCI Object Storage.

Allocating Notebook Session Swap Space

Swap space is allocated on disk and is used as memory when the RAM is full. Space on the attached Block Volume is allocated for swap space for a notebook session. The swap space you allocate to the notebook is the minimum of the memory-configuration of the Compute shape you selected or 25% of the total Block Volume storage.

For example, if the memory of the Compute shape is 16 GB and the Block Volume is 100 GB, then the swap space allocated to the notebook is 16 GB. This is because 16 GB is smaller than 25 GB, which is 25% of the total Block Volume.

Another example is if the Compute shape memory is 32 GB and the Block Volume is 100 GB, then the notebook session would be allocated 25 GB of swap space.

Note

Spillage into swap space can result in massively slow performance due to page swapping. We recommend that you find ways to design your code to better manage your memory usage.

Managing Notebook Session Timeout

You are automatically logged out of a notebook session after an hour of inactivity by default. You can change this behavior in each notebook and extend the notebook session time by up to 23 hours for a total of 24 hours.

To change the notebook timeout, click Session remaining: <hours minutes> and select the number of hours to extend the timeout.

The countdown timer indicates how much time is left before the automatic logout. The amount of time before logout can change depending on your activity level and idle time whether you use the default or change the timeout. You are logged out when the timer is at zero. To reset the timer, you have to log out then log back into the notebook.

Downloading a Dataset

You can download a dataset file (.csv) from a Data Science notebook session using the JupyterLab download option.

Locate and select the dataset you want to download. Right-click the file, and select Download. If your browser can open the file, the file is opened in a separate browser tab. Otherwise, the file is downloaded directly to your disk. This behavior is controlled by your browser settings.

Tip

You can download other files from JupyterLab such as notebook example (.ipynb) files.

Moving Notebook Sessions

You can move a notebook session from its current compartment to a different one.

For example, you may want to move a notebook session to promote it from a development compartment to production compartment, or you could change the visibility of the notebook session.

You can either click the notebook session's name or the Actions menu, and click Move resource. Select the destination compartment and click Move resource.

Deactivating and Activating Notebook Sessions

Deactivating Notebook Sessions

Before you deactivate a notebook session, save your work to its attached Block Volume to retain your data. Any data on the boot volume of the notebook session is deleted after you deactivate it.

You can either click the notebook session's name or the Actions menu, and click Deactivate to deactivate a notebook session after you provide confirmation.

When you deactivate a notebook session, the underlying Compute stops, while the attached Block Volume is detached and preserved. The Block Volume drive is mounted in /home/datascience so any data, notebooks, and conda environments you save are stored there. The work you save in the Block Volume drive is retained after a notebook session is deactivated. When a notebook session is reactivated, you can access the work you previously saved in the Block Volume.

Activating Notebook Sessions

You can either click the notebook session's name or the Actions menu, and click Activate to activate a notebook session after you provide confirmation. When you activate a notebook session, a new Compute shape is provisioned. The Block Volume from the previous notebook session is restored and attached to the new Compute shape.

When you activate a notebook session, any previously saved data or files on the block volume of that deactivated notebook session are available in the activated notebook session.

Caution

When activating a notebook session, you can only choose a VCN and subnet that allows your reactivated notebook session to remain in the same Availability Domain (AD). If you choose an AD-specific subnet that would change the AD of the notebook session, then the activation action fails because the Block Volume reattachment fails. We generally recommend that you use regional subnets to avoid this type of error.

While activating a notebook session, you can change the previously configured Compute shape, Block Volume size, VCN, and subnet.

Note

If you choose to change the Block Volume size, then you must specify a larger value than the current Block Volume size.

Deleting Notebook Sessions

You can either click the notebook session's name or the Actions menu, and click Delete to delete a notebook session.

Enter the name exactly as it appears because it's case sensitive, then click Delete.

The notebook session’s underlying Compute is stopped, and the Block Volume is deleted. Deactivating Notebook Sessions explains how to retain Block Volume data.

Note

Deleted notebook sessions exist in lists for 30 days after being deleted. You can filter them out of lists using the State filter.

The delete notebook session action is asynchronous and initiates a work request. You can use the work request to track the status of an operation. For general information about using work requests in OCI, see Work Requests and the Work Requests API.

Caution

If you want to keep the file changes in a notebook session’s file system and attached Block Volume, you must back it up before you delete the notebook session. For example, you can copy files to an object storage bucket, or commit and push changes to a Git repository outside the notebook session. Otherwise, all file changes in the notebook session’s file system and attached Block Volume are lost.

Accessing the Internet from Notebook Sessions

When you create a notebook session, you choose to use a public or private subnet. Either subnet must have a Network Address Translation (NAT) gateway and matching route table, and security list rules to achieve egress to the internet. We recommend that you use a private subnet with a route to a NAT gateway for internet access for security.

To create a NAT gateway, you first create a Virtual Cloud Network, and then you set up your NAT gateway.

Notebook Session Metrics

Metrics are automatically available for any Data Science notebook session that you create in the oci_datascience namespace. You must enable monitoring on OCI resources to get these metrics.

Data Science notebook session metrics include these dimensions:

RESOURCEID

The OCID of the notebook session.

RESOURCEDISPLAYNAME

The user-supplied display name of the notebook session.

SHAPE

The Compute instance shape of the notebook session.

Metric Name Display Name Unit Description Dimensions
CpuUtilization CPU Utilization Percent Activity level from the CPU. Expressed as a percentage of the busy time compared with the total time (busy and idle).

resourceId

resourceDisplayName

shape

MemoryUtilization Memory Utilization Percent Space currently in use. Measured by pages. The percentage of RAM that is used. It's measured by the number of memory pages used relative to the total number of memory pages.

resourceId

resourceDisplayName

shape

NetworkBytesIn Network Bytes In Byte Network receipt throughput. Expressed as bytes per second.

resourceId

resourceDisplayName

shape

NetworkBytesOut Network Bytes Out Byte Network transmission throughput. Expressed as bytes transmitted per second.

resourceId

resourceDisplayName

shape

Viewing Notebook Session Metrics

You can view the default metric charts using a notebook session details page in the Data Science service.

  1. Open the navigation menu and click Analytics & AI. Under Machine Learning, click Data Science.
  2. Select the region you are using with Data Science.
  3. Select the compartment that contains the project of the notebook session that you want to view the metrics for.
  4. Click the name of the project associated with the notebook session that you want to view the metrics for.
  5. Click the name of the notebook session that you want to view metrics for.
  6. Under Resources, click Metrics.

    The Metrics page displays a chart for each metric that is emitted by the metric namespace for Data Science.

For more information about the emitted metrics, see notebook session metrics.

If you don't see the metrics data for a notebook session that you expect, see missing metrics data.

Using the API