What are... Magic commands?

Some of you might be familiar with magic commands from iPython. Well Databricks notebooks have the same built-in utility. These %magic commands can range from the more functional, such as running shell commands (%sh) or interacting with the Databricks file system (%fs), to the more insightful (see %who and %config). Ultimately they are a powerful coding tool that most users can take advantage of in the majority of contexts.

Most importantly, %lsmagic cmd will list all the available magic commands.

Below are some of the more common %magic commands that can be run from the start of each cell in a Databricks notebook:

%run: allows you to include and run another notebook or python file in your current notebook. Be careful as there is some nuance to this when it comes to modularization and jobs scheduling.
%sh: allows you to execute shell commands in your the cluster nodes.
%fs: allows you to interact with the Databricks file system.
%sql: allows you to run SQL queries.
%scala: switches the notebook context to Scala.
%python: switches the notebook context to Python.
%md: allows you to write markdown text.
%r: switches the notebook context to R.
%jobs: lists all the running jobs.
%config: allows you to set configuration options for the notebook.
%reload: reloads the contents of a module.
%pip: allows you to install Python packages.
%load: loads the contents of a file into a cell.
%matplotlib: sets up the matplotlib backend.
%who: lists all the variables in the current scope.
%env: allows you to set environment variables

Source: https://docs.databricks.com/en/notebooks/notebooks-code.html#develop-code-in-databricks-notebooks