Finding the Path to a Managed Table in Databricks

Photo by Anisur Rahman on Unsplash

This article shows how to find a path for a managed Databricks table.

In Databricks, you might have been creating managed tables, writing to managed tables and reading from managed tables using the database.tablename (catalog.database.tablename, if you have upgraded to Unity Catalog) pattern. And sometimes you may need to get the path to the table similar to the paths for external tables that use abfss://deltatable@resourcename.dfs.core.windows.net/container/tablename pattern, if you are using Azure for example.

By default, managed tables are stored in /user/hive/warehouse in Hive warehouse. The table you are interested in is located under /user/hive/warehouse directory with table_name.db/table_name. For example, /user/hive/warehouse/your_table_name.db/your_table_name.

To build the path, you can add dbfs: to the beginning of /user/hive/warehouse/your_table_name.db/your_table_name

or run dbutils.fs.ls("/user/hive/warehouse/your_table_name.db/your_table_name") which will displaypath='dbfs:/user/hive/warehouse/your_table_name.db/your_table_name'

You can also go to actual table data location following the path by clicking on the Data tab on the left side in Databricks workspace, then DBFS button on the top, then user folder, then hive folder and then warehouse folder, where you will find all your managed tables.

Alternative, you can run

DESCRIBE EXTENDED your_table_name

SQL command. If you are running your notebook in Python, you will need to run “%sql” magic command before the SQL command to execute.

Or run

DESCRIBE DETAIL your_table_name

which will have “Location” field that shows the path of your managed table. The above command will show your table location if your table was in the default database or if you have only one database or if you have already set “USE DATABASE your_database_name” command. If you have to specify the database to your table, you can set database to use first and then run your commands together like in the code snipped below.

USE DATABASE your_database_name;
DESCRIBE EXTENDED your_table_name
--OR
DESCRIBE DETAIL your_table_name