dbutils.widgets.removeAll()
A Databricks notebook is an interactive, multi-language document that combines executable code, visualisations, and documentation in a single shareable artifact. Creating one takes seconds — attach it to compute, write your first cell, and press Shift+Enter to see results immediately.
Who this is for:
Part of the Getting Started with Databricks section of the Databricks tutorial series.
Architecture / Concept Overview: dbutils.widgets.removeAll()
Notebooks in Databricks execute code against a Spark cluster or SQL warehouse. Each notebook has a default language but can mix languages using magic commands. The notebook runtime maintains state between cells, so variables persist across executions within a session.
*Figure 1 — Notebook execution flow: your code runs on an attached cluster and returns results inline.*
*Figure 2 — Notebook cells share session state; variables from one cell are available in subsequent cells.*
*Figure 3 — Notebook lifecycle: develop interactively, share with colleagues, then schedule for production.*
Key Terms
Prerequisites and Setup
- An active Databricks workspace with login access
- A running cluster or permission to create one (or a SQL warehouse for SQL-only notebooks)
- Basic familiarity with Python or SQL syntax
- Understanding of what you want to explore (a dataset, a sample, or an experiment)
Step-by-Step Implementation
Configuration Reference
| Feature | Shortcut / Method | Description |
|---|---|---|
| Run cell | Shift + Enter | Execute current cell and advance |
| Run cell in place | Ctrl + Enter | Execute without advancing |
| Add cell above | A (in command mode) | Insert new cell above current |
| Add cell below | B (in command mode) | Insert new cell below current |
| Delete cell | D, D (in command mode) | Delete current cell |
| Toggle comment | Ctrl + / | Comment/uncomment selected lines |
| Auto-complete | Tab | Trigger code completion |
| Format code | Ctrl + Shift + F | Auto-format cell contents |
Monitoring, Cost, and Security Considerations
Monitoring
Monitor cluster utilisation while running notebooks via the Spark UI (accessible from the cluster page). Check cell execution times to identify slow operations. Use the notebook revision history to track changes over time.
Cost Optimisation
Detach from the cluster when not actively working (or rely on auto-termination). Avoid running display() on very large DataFrames — add a .limit() first. Use %sql cells with SQL warehouses instead of spinning up full clusters for simple queries.
Security and Governance
Do not hardcode credentials in notebook cells — use Databricks secrets (dbutils.secrets.get()). Be mindful of notebook sharing permissions — shared notebooks expose all cell content. Clear cell outputs containing sensitive data before sharing.
Common Pitfalls and Recommended Patterns
- Running cells out of order and encountering undefined variables — use "Run All" to verify sequential execution
- Forgetting to attach a cluster before running — the notebook will prompt you, but it causes a delay
- Using
print()for DataFrames instead ofdisplay()— display provides richer formatting and visualisation options - Not using magic commands — writing SQL in a Python cell causes syntax errors; prefix with
%sql - Creating huge DataFrames with
toPandas()— this collects all data to the driver; add.limit()for exploration - Not documenting with markdown cells — notebooks become unreadable without context
Frequently Asked Questions
Can multiple people edit the same notebook?
Yes. Databricks supports real-time co-authoring. You'll see other users' cursors and edits live, similar to Google Docs.
Are notebook changes saved automatically?
Yes. Databricks auto-saves notebook changes. You can also view and restore any previous version through revision history.
Can I run a notebook as a scheduled job?
Yes. Any notebook can be scheduled as a Databricks job. Navigate to Workflows, create a job, and select the notebook as the task source.
How do I install additional Python packages?
Use %pip install package_name in a cell. The package installs on the cluster for the current session. For persistent packages, use the cluster's Libraries configuration.
Can I export notebooks to other formats?
Yes. Export as .py (Python script), .ipynb (Jupyter), .dbc (Databricks archive), or HTML. Use File > Export from the notebook menu.