SkyPilot¶

SkyPilot is a framework tailored for executing Large Language Models (LLMs) across various cloud platforms, ensuring optimal cost efficiency, abundant GPU availability, and streamlined execution management. It assists in provisioning tasks to utilize the most advantageous resources available while minimizing expenses.

Prerequisites¶

You must have a SkyPilot-configurable cloud account.

Note

Currently SkyPilot supports the following cloud infrastructure: Kubernetes; AWS, GCP, Azure, OCI, Lambda Cloud, Fluidstack, RunPod, Cudo, Paperspace, Cloudflare, Samsung, IBM, VMware vSphere. For more information, visit the SkyPilot documentation
If you are in an organization, an IAM User Account is needed to set up SkyPilot policy for the account.
- For example; If you have an AWS IAM user account, use the following URL:
  
  Skypilot- Minimal Permissions
Also, in case of using AWS cloud account, an Access Key and Secret Access Key is required to access the AWS account resources.
- To obtain the access key and secret access key, click on the following page and follow the steps provided:
  
  AWS- Managing access keys (console)

Configuring SkyPilot on DKubeX¶

To set up SkyPilot on your DKubeX setup, use the following steps.

On your DKubeX backend, execute into the sky-controller-v2-0 pod.
```
kubectl exec -it -n d3x sky-controller-v2-0 -- bash
```
Use the following command and provide your AWS access key and secret key. Rest of the fields can be left blank as default.
```
aws configure
```

Workflow Tutorials with SkyPilot¶

Visit the links provided below for detailed tutorials regarding running specific workflows in DKubeX with SkyPilot.

Deploying Embedding Models with SkyPilot

Tutorial on deploying embedding models on DKubeX with SkyPilot and Sky-Serve.

./emb-deployment-with-skypilot.html

Deploying LLMs with SkyPilot

Tutorial on how to deploy LLMs on DKubeX with SkyPilot and Sky-Serve.

./llm-deployment-with-skypilot.html

Wine and Llama-2 Finetuning with SkyPilot

Tutorial on running Sky jobs regarding finetuning Wine and Llama-2 models on DKubeX.

./wine-and-llama2-finetuning-with-skypilot.html