SkyPilot

SkyPilot is a framework tailored for executing Large Language Models (LLMs) across various cloud platforms, ensuring optimal cost efficiency, abundant GPU availability, and streamlined execution management. It assists in provisioning tasks to utilize the most advantageous resources available while minimizing expenses.

Prerequisites

• You must have a SkyPilot-configurable cloud account.

  Note

  Currently SkyPilot supports the following cloud infrastructure: Kubernetes; AWS, GCP, Azure, OCI, Lambda Cloud, Fluidstack, RunPod, Cudo, Paperspace, Cloudflare, Samsung, IBM, VMware vSphere. For more information, visit the SkyPilot documentation

• If you are in an organization, an IAM User Account is needed to set up SkyPilot policy for the account.

  • For example; If you have an AWS IAM user account, use the following URL:

    Skypilot- Minimal Permissions

• Also, in case of using AWS cloud account, an Access Key and Secret Access Key is required to access the AWS account resources.

  • To obtain the access key and secret access key, click on the following page and follow the steps provided:

    AWS- Managing access keys (console)

Configuring SkyPilot on DKubeX

To set up SkyPilot on your DKubeX setup, use the following steps.

• Go to the admin page of your DKubeX setup. To access the Admin page, add /admin at the end of your DKubeX setup URL.

  Example: If your DKubeX setup URL is;

  https://a1183e6450d4e4d138415gaf0d11b362-62d15cf634758ebd.elb.us-east-1.amazonaws.com

  Then, to access the Admin page, add /admin at the end as shown below;

  https://a1183e6450d4e4d138415gaf0d11b362-62d15cf634758ebd.elb.us-east-1.amazonaws.com/admin

• Log in with your DKubeX admin credentials.

• Open the Settings tab on the top of the page, and click on the Cloud Provider option to open the cloud provider settings.

• Choose your prefered cloud provider service from the dropdown provided. Once done, provide your credentials for that particular cloud provider.

• Once you have provided the necessary credentials, click on the Save button to save the changes.

Workflow Tutorials with SkyPilot

Visit the links provided below for detailed tutorials regarding running specific workflows in DKubeX with SkyPilot.

Deploying Embedding Models with SkyPilot

Tutorial on deploying embedding models on DKubeX with SkyPilot and Sky-Serve.

./emb-deployment-with-skypilot.html

Deploying LLMs with SkyPilot

Tutorial on how to deploy LLMs on DKubeX with SkyPilot and Sky-Serve.

./llm-deployment-with-skypilot.html

Wine and Llama-2 Finetuning with SkyPilot

Tutorial on running Sky jobs regarding finetuning Wine and Llama-2 models on DKubeX.

./wine-and-llama2-finetuning-with-skypilot.html