1. Packages
  2. Azure Classic
  3. How-to Guides
  4. Spark on Azure HDInsight

We recommend using Azure Native.

Azure Classic v6.3.1 published on Tuesday, Oct 1, 2024 by Pulumi

Spark on Azure HDInsight

azure logo

We recommend using Azure Native.

Azure Classic v6.3.1 published on Tuesday, Oct 1, 2024 by Pulumi

    View Code Deploy this example with Pulumi

    An example Pulumi component that deploys a Spark cluster on Azure HDInsight.

    Running the App

    1. Create a new stack:

      $ pulumi stack init dev
      
    2. Login to Azure CLI (you will be prompted to do this during deployment if you forget this step):

      $ az login
      
    3. Restore NPM dependencies:

      $ npm install
      
    4. Run pulumi up to preview and deploy changes:

      $ pulumi up
      Previewing changes:
      ...
      
      Performing changes:
      ...
      info: 5 changes performed:
          + 5 resources created
      Update duration: 15m6s
      
    5. Check the deployed Spark endpoint:

      $ pulumi stack output endpoint
      https://myspark1234abcd.azurehdinsight.net/
      
      # For instance, Jupyter notebooks are available at https://myspark1234abcd.azurehdinsight.net/jupyter/
      # Follow https://docs.microsoft.com/en-us/azure/hdinsight/spark/apache-spark-load-data-run-query to test it out
      
    azure logo

    We recommend using Azure Native.

    Azure Classic v6.3.1 published on Tuesday, Oct 1, 2024 by Pulumi