Deploying models from MLFlow

The registered models in the Models page in MLFlow in DKubeX can be deployed.

  • On the DKubeX Workspace, click on the terminal application to open the DKubeX CLI on a new tab.

  • Type the following command while replacing with the appropriate details enclosed in hashes.

d3x serve create -n <deployment name> -r mlflow --model <model name> --model_version <model version number> --repo_name <name of repo containing serving script> --repo_org <username of repo owner> --is_private_repo --access_token <your github personal access token> --branch_name <name of the branch containing the serving script> --depfilepath <deployment file path> --ngpus <number of gpu> --min_cpu <min number of cpus for cluster creation> --max_cpu <max number of cpus for cluster creation> --min_memory <min memory for cluster creation> --max_memory <max memory for cluster creation> --min_replicas <min number of deployment replicas> --max_replicas <max number of deployment replicas>

Note

  • In the serving command, if the values of hardware type, cpu, gpu, memory and replica are not provided, it automatically accepts the default values for them.

  • Use –is_private_repo and –access_token flags if the repository containing the serving script is private. Also make sure that you have access to the repository containing the serving script.

  • To find the model name and version number, use the following steps-

    • Open the MLFlow tab on DKubeX UI.

      ../_images/mlflowexperiments0.png

    • Open the Models page in MLFlow. This will show the list of all registered models.

      ../_images/mlflowmodels1.png

    • Note down the name and the version number of the model you are going to deploy.

  • After entering all necessary details, press Enter. The deployment creation will start automatically.

  • On the DKubeX UI, go to the Deployments tab. The list on the screen should show your deployment, along with it’s status, the name of the serve cluster and the serve endpoint.

    • The endpoint will show up after the deployment is successful, and the status turns to running.

    • Clicking on the name of the deployment opens the page containing all the details of that particular deployment.

    • Clicking on the name of the serve cluster opens the Ray dashboard for that cluster.

      ../_images/mlflowdeploy4.png