Databricks

databricks is a unified data analytics platform for running sql queries, managing jobs, and working with pipelines and file storage with the databricks app in make, you can manage your databricks jobs, pipelines, and unity catalog volumes in your databricks account the databricks app is only available on the make enterprise plan https //www make com/en/enterprise requirements to use the databricks app in make, you must have a databricks account https //www databricks com/ with a service principal for subscription information, refer to the databricks pricing page https //www databricks com/product/pricing this connection uses a databricks service principal to access resources such as jobs, pipelines, catalogs, or unity catalog volumes, the service principal must be granted permissions on each resource for more information, see the databricks oauth m2m documentation https //docs databricks com/aws/en/dev tools/auth/oauth m2m connect databricks and to get started, you must first create a connection between databricks and , allowing the two services to communicate to create the connection, you need to obtain your databricks credentials docid\ ye9hra1yw2c je4efp8ma create the connection in make docid\ ye9hra1yw2c je4efp8ma obtain your credentials in databricks before you create the connection in make, you must obtain your databricks workspace url and service principal credentials to obtain your credentials log in to your databricks account copy your workspace url from the browser address bar and store it in a safe place the url follows the following format https //dbc a1234567 abcd cloud databricks com go to settings > identity and access > service principals select your service principal copy the application id value this is your client id store it in a safe place in the secrets tab, generate or copy your oauth secret this is your client secret store it in a safe place you'll use these values in the create a connection dialog in make create the connection in once you have your databricks credentials, you're ready to create the connection in make to create the connection log in to your account, add a databricks module to your , and click create a connection optional in the ​​ connection name​​ field, enter a name for the connection in the ​​ workspace url field, enter your databricks workspace url copied above for example, https //dbc a1234567 abcd cloud databricks com in the client id field, enter the application id of your databricks service principal in the client secret field, enter the oauth secret of your databricks service principal click save if prompted, authenticate your account and confirm access you've successfully created the connection and can now use the databricks app in your if you want to make changes to your connection in the future, follow the steps here https //help make com/connect an application#tn2pj databricks modules after connecting to the databricks app, you can use the following types of modules to build your sql execute a sql statement get a sql statement result jobs run a job get a job run output pipelines get a pipeline update status files upload a file to a unity catalog volume templates you can look for databricks templates in make's template gallery https //www make com/en/templates , where you'll find thousands of pre created databricks resources databricks api documentation https //docs databricks com/api/workspace/introduction databricks help center https //help databricks com/s/