d3x Commands

d3x commands are used for managing the user workspace, apikeys, ray clusters, user apps and deployments. This page provides information about all of the d3x commands that are currently available, along with their functions, usage syntax and available options.

Usage

CommandExample

d3x [OPTIONS] COMMAND [ARGS]...

Function

Group for d3x commands.

Options

Option	Description
-d, --debug	Enable requests debugging.
--profile TEXT	Cluster profile.
--help	Show help for the command.

Commands

Command	Description
about	Details about DKubeX CLI config
apikey	Commands to generate and get DKubeX API keys.
apps	Commands regarding creating and managing user-defined applications.
backup	Commands for backing up datasets.
dataset	Commands for creating, managing and using datasets.
emb	Commands regarding using embedding models.
fm	Commands regarding using foundation models.
ft	LLM Finetuning process related commands.
instance-types	List available instance types.
llms	Commands regarding using LLMs.
login	Login to DKubeX with particular user profile in the CLI.
mlflow	Commands for using and managing experiments and models in MLFlow from CLI.
mount	Mount S3 bucket in read/write mode.
profile	Commands regarding user profiles in DKubeX.
ray	Commands regarding Ray jobs and Ray clusters.
restore	Commands for restoring databases.
serve	Commands for creating model deployments.
sky	Commands for managing SkyPilot clusters, resources and jobs.
umount	Unmount bucket previously mounted using d3x mount command.
version	Show DKubeX version.
ws	Commands for managing workspaces and workspace applications.

d3x about

• Usage

  CommandExample

  d3x about [OPTIONS]
  

• Function

  Details about d3x CLI config

• Options

  Option	Description
  -o, --output [yaml|json]	Output format (supported- .json, .yaml)
  --help	Show help for the command.

d3x apikey

• Usage

  CommandExample

  d3x apikey [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands to generate and get DKubeX API keys.

• Options

  Option	Description
  --help	Show help for the command.

• Commands

  Command	Description
  generate	Generate new API key for the user.
  get	Get currect API key for the user.

  d3x apikey generated3x apikey get

  • Usage

    CommandExample

    d3x apikey generate [OPTIONS]
    

    • Function

      Generate new API key for the user.

    • Options

      Option	Description
      -o, --output [yaml|json]	Output format (supported- .json, .yaml)
      --help	Show help for the command.

d3x apps

• Usage

  CommandExample

  d3x apps [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands regarding creating and managing user-defined applications.

• Options

  Option	Description
  --help	Show help for this command.

• Commands

  Command	Description
  create	Deploy the user-defined application on DKubeX.
  delete	Delete user apps.
  list	List user apps.
  open	Open user app in browser.
  start	Start user apps.
  stop	Stop user apps.

  d3x apps created3x apps deleted3x apps listd3x apps opend3x apps startd3x apps stop

  • Usage

    CommandExample

    d3x apps create [OPTIONS] [COMMAND]...
    

    • Function

      Deploy the user-defined application on DKubeX.

    • Options

      Option	Description
      -c, --config TEXT	Path to the .yaml configuration file
      --help	Show help for this command.

d3x backup

• Usage

  CommandExample

  d3x backup [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for backing up datasets.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  weaviate	command for weaviate backup

  d3x backup weaviate

  • Usage

    CommandExample

    d3x backup weaviate [OPTIONS]
    

  • Function

    Command for dataset backup to Weaviate.

  • Options

    Option	Description
    -d, --dataset TEXT	A name to represent the ingested docs [required]
    -i, --id TEXT	An ID for the backup [required]
    -b, --backend [filesystem|s3]	s3 or filesystem to save the backup [required]
    --help	Show help for the command.

d3x dataset

• Usage

  CommandExample

  d3x dataset [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for creating, managing and using datasets.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  delete	Deletes specified dataset if present from the database
  evaluate	Run evaluation pipeline over the given dataset.
  ingest	Ingest the documents from provided source into the database.
  list	lists all the datasets present in database
  query	Query over documents using the selected pipeline.
  show	shows documents present in the specified dataset

  d3x dataset deleted3x dataset evaluated3x dataset ingestd3x dataset listd3x dataset queryd3x dataset show

  • Usage

    CommandExample

    d3x dataset delete [OPTIONS] DATASET
    

  • Function

    Deletes specified dataset if present from the database

  • Options

    Option	Description
    --help	Show help for the command..

d3x emb

• Usage

  CommandExample

  d3x emb [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commads regarding using embedding models.

• Options

  Option	Description
  --help	Show help for the command.

• Commands

  Command	Description
  deploy	Command to deploy an embedding model.
  list	List the supported embedding models on DKubeX.

  d3x emb deployd3x emb list

  • Usage

    CommandExample

    d3x emb deploy [OPTIONS]
    

  • Function

    Command to deploy an embedding model.

  • Options

    Option	Description
    -n, --name TEXT	name of the deployment [required]
    -m, --model TEXT	name or path of the model to deploy for embeddings
    --base_model TEXT	base model name
    --token TEXT	hugging face token
    --config TEXT	github raw content url or local config path
    --mlflow TEXT	name of the mlflow registered model along with version (example: diffusion:1)
    --image TEXT	customized docker image for llm deployment
    --type TEXT	instance type
    --min_replicas TEXT	minimum replicas
    --max_replicas TEXT	maximum replicas
    --publish	publish the deployment
    -o, --output [yaml|json]	supported only json,yaml
    -sky, --remote-sky	If the model should be deployed on remote sky cluster.
    --sky-accelerator TEXT	accelerator for remote sky cluster
    --sky-yaml TEXT	Path to custom yaml file for sky serve.
    --help	Show help for the command.

d3x fm

• Usage

  CommandExample

  d3x fm [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands regarding using foundation models.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  delete	Delete FM job.
  get	Get FM job details.
  jobs	List FM jobs.
  logs	Show logs of given FM job.
  trainchunks	Create train and/or test chunks for finetuning.

  d3x fm deleted3x fm getd3x fm jobsd3x fm logsd3x fm trainchunks

  • Usage

    CommandExample

    d3x fm delete [OPTIONS] JOB_NAME
    

  • Function

    Delete FM job.

  • Options

    Option	Description
    --help	Show help for the command..

d3x ft

• Usage

  CommandExample

  d3x ft [OPTIONS] COMMAND [ARGS]...
  

• Function

  LLM Finetuning process related commands.

• Options

  Option	Description
  --help	Show help for the command.

• Commands

  Command	Description
  delete	delete finetuning jobs
  details	finetuning job details
  finetune	finetuning of model
  get-config	get predefined finetuning config
  jobs	list finetuning jobs
  list-configs	list predefined finetuning configs
  logs	finetuning job logs
  merge	create finetuning job
  quantize	create finetuning job

  d3x ft deleted3x ft detailsd3x ft finetuned3x ft get-configd3x ft jobsd3x ft list-configsd3x ft logsd3x ft merged3x ft quantize

  • Usage

    CommandExample

    d3x ft delete [OPTIONS] NAME
    

  • Function

    delete finetuning jobs

  • Options

    Option	Description
    --help	Show help for the command.

d3x llms

• Usage

  CommandExample

  d3x llms [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands regarding using LLMs.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  deploy	Deploy a LLM on dkubex.
  list	List the supported LLMs

  d3x llms deployd3x llms list

  • Usage

    CommandExample

    d3x llms deploy [OPTIONS]
    

  • Function

    Deploy a LLM on dkubex.

  • Options

    Option	Description
    -n, --name TEXT	name of the deployment [required]
    -m, --model TEXT	name or path of the model to deploy for serving
    --base_model TEXT	base model name
    --token TEXT	hugging face token
    --config TEXT	github raw content url or local config path
    --mlflow TEXT	name of the mlflow registered model along with version (example: diffusion:1)
    --image TEXT	customized docker image for llm deployment
    --type TEXT	instance type
    --min_replicas TEXT	minimum replicas
    --max_replicas TEXT	maximum replicas
    --publish	publish the deployment
    -o, --output [yaml|json]	supported only json,yaml
    -sky, --remote-sky	If the model should be deployed on remote sky cluster.
    --sky-accelerator TEXT	accelerator for remote sky cluster
    -k, --dkubex-apikey TEXT	API key for remote clusters to reach dkubex.
    --sky-yaml TEXT	Path to custom yaml file for sky serve.
    --help	Show help for the command..

d3x login

• Usage

  CommandExample

  d3x login [OPTIONS]
  

• Function

  Login to DKubeX with particular user profile in the CLI.

• Options

  Option	Description
  --help	Show help for the command.

d3x mlflow

• Usage

  CommandExample

  d3x mlflow [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for using and managing experiments and models in MLFlow from CLI.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  experiments	Commands for managing experiments in MLFlow.
  models	Commands for managing models in MLFlow.

  d3x mlflow experimentsd3x mlflow models

  • Usage

    CommandExample

    d3x mlflow experiments [OPTIONS] COMMAND [ARGS]...
    

  • Function

    Commands for managing experiments in MLFlow.

  • Options

    Option	Description
    --help	Show help for the command..

  • Commands

    Command	Description
    archive	Archive experiment in MLFlow
    delete	Delete experiment in MLFlow
    list	List experiments in MLFlow
    restore	Restore experiment in MLFlow

    d3x mlflow experiments archived3x mlflow experiments deleted3x mlflow experiments listd3x mlflow experiments restore

    • Usage

      CommandExample

      d3x mlflow experiments archive [OPTIONS] ID
      

    • Function

      Archive experiment in MLFlow.

    • Options

      Option	Description
      --help	Show help for the command..

d3x mount

• Usage

  CommandExample

  d3x mount [OPTIONS] [BUCKET] [MOUNTPATH]
  

• Function

  Mount S3 bucket in read/write mode.

• Options

  Option	Description
  -a, --all	Mounts all the previous buckets.
  -d, --dry-run	Runs a dry run of mounting bucket, does not mount the buckets.
  --help	Show help for the command.

d3x profile

• Usage

  CommandExample

  d3x profile [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands regarding user profiles in DKubeX.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  configure	Configure user profile.
  delete	Delete user profile.
  list	List user profiles.
  set	Set default user profile.

  d3x profile configured3x profile deleted3x profile listd3x profile set

  • Usage

    CommandExample

    d3x profile configure [OPTIONS]
    

  • Function

    Configure user profile.

  • Options

    Option	Description
    --help	Show help for the command..

d3x ray

• Usage

  CommandExample

  d3x ray [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands regarding Ray jobs and Ray clusters.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  activate	Set Ray cluster as active cluster.
  address	List Ray clusters along with their cluster Ray address.
  create	Create Ray cluster.
  dashboard	Open Ray dashboard.
  delete	Delete Ray cluster.
  get	Get a Ray cluster.
  job	Commands regarding managing Ray jobs in currently active Ray cluster.
  list	List Ray clusters.

  d3x ray activated3x ray addressd3x ray created3x ray dashboardd3x ray deleted3x ray getd3x ray jobd3x ray list

  • Usage

    CommandExample

    d3x ray activate [OPTIONS] CLUSTER
    

  • Function

    Set Ray cluster as active cluster.

  • Options

    Option	Description
    -o, --output [yaml|json]	supported only json,yaml
    --help	Show help for the command..

d3x restore

• Usage

  CommandExample

  d3x restore [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for restoring databases.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  weaviate	Command for Weaviate restore.

  d3x restore weaviate

  • Usage

    CommandExample

    d3x restore weaviate [OPTIONS]
    

  • Function

    Command for Weaviate restore.

  • Options

    Option	Description
    -i, --id TEXT	An ID for the backup [required]
    -b, --backend [filesystem|s3]	s3 or filesystem to save the backup [required]
    --help	Show help for the command..

d3x serve

• Usage

  CommandExample

  d3x serve [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for creating model deployments.

• Options

  Option	Description
  --help	Show help for the command..

• Commands

  Command	Description
  create	Create deployment
  delete	Delete deployment
  get	Get Deployment
  list	List Deployments
  status	Check deployment Status

  d3x serve created3x serve deleted3x serve getd3x serve listd3x serve status

  • Usage

    CommandExample

    d3x serve create [OPTIONS]
    

  • Function

    Create deployment.

  • Options

    Option	Description
    -n, --name TEXT	name of the ray serve [required]
    -r, --registry_type [hugging_face|mlflow|local|none]	registry type [required]
    --hface_repoid TEXT	hugging face repo id
    --hface_tokenizer TEXT	hugging face tokenizer
    --hface_classifier TEXT	hugging face classifier
    --hface_token TEXT	hugging face token
    --model TEXT	mlflow model name
    --model_version TEXT	mlflow model version
    --depfilepath TEXT	deployment filepath
    --working_dir TEXT	local serving directory path
    --is_private_repo	private repo
    --repo_name TEXT	github repo name
    --repo_org TEXT	github personal/organization account
    --access_token TEXT	github access token
    --branch_name TEXT	github branch name
    --commit_id TEXT	github commit id
    --unprotected	serving without token
    -e, --env TEXT	environment variable
    --ngpus TEXT	number of gpus
    --min_cpu TEXT	min number of cpus for cluster creation
    --max_cpu TEXT	max number of cpus per cluster creation
    --min_memory TEXT	min memory for cluster cluster creation
    --max_memory TEXT	max memory for cluster cluster creation
    --min_replicas TEXT	min deployment replicas
    --max_replicas TEXT	max deployment replicas
    --max_concurrent_requests TEXT	max concurrent requests
    --description TEXT	deployment description
    --image TEXT	cluster image
    --publish	publish the deployment
    --no_rsrc_limits	no resource limits
    --type TEXT	node selector
    --timeout INTEGER	deployment timeout
    --kserve	enable kserve deployment
    --model_format [tensorflow|sklearn|pytorch|xgboost|pmml|lightgbm|paddle|mlflow|custom]	model format in case of kserve deployment
    --model_path TEXT	local path of model for kserve deployment
    --protocol_version [v1|v2]	dataplane protocol
    -o, --output [yaml|json]	supported only json,yaml
    --help	Show help for the command..

d3x sky

• Usage

  CommandExample

  d3x sky [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands for managing SkyPilot clusters, resources and jobs.

• Options

  Option	Description
  --install-shell-completion [bash|zsh|fish|auto]	Install shell completion for the specified shell.
  --uninstall-shell-completion [bash|zsh|fish|auto]	Uninstall shell completion for the specified shell.
  -v, --version	Show the version and exit.
  -c, --commit	Show the commit hash and exit
  -h, --help	Show help for the command..

• Commands

  Command	Description
  launch	Launch a cluster or task.
  exec	Execute a task or command on an existing cluster.
  status	Show clusters.
  cost-report	Show estimated costs for launched clusters.
  queue	Show the job queue for cluster(s).
  logs	Tail the log of a job.
  cancel	Cancel job(s).
  stop	Stop cluster(s).
  autostop	Schedule an autostop or autodown for cluster(s).
  start	Restart cluster(s).
  down	Tear down cluster(s).
  check	Check which clouds are available to use.
  show-gpus	Show supported GPU/TPU/accelerators and their prices.
  storage	Commands for using SkyPilot Storage CLI.
  bench	Commands for usingSkyPilot Benchmark CLI.
  jobs	Commands for using SkyPilot managed Jobs CLI.
  spot	Commands for using managed Spot CLI (spot instances with auto-recovery).
  serve	Commands for using SkyServe CLI (multi-region, multi-cloud serving).

  d3x sky launchd3x sky execd3x sky statusd3x sky cost-reportd3x sky queued3x sky logsd3x sky canceld3x sky stopd3x sky autostopd3x sky startd3x sky downd3x sky checkd3x sky show-gpusd3x sky storaged3x sky benchd3x sky jobsd3x sky spotd3x sky serve

  • Usage

    CommandExample

    d3x sky launch [OPTIONS] [ENTRYPOINT]...
    

  • Function

    Launch a cluster or task. If ENTRYPOINT points to a valid YAML file, it is read in as the task specification. Otherwise, it is interpreted as a bash command. In both cases, the commands are run under the task’s workdir (if specified) and they undergo job queue scheduling.

  • Options

    Option	Description
    -c, --cluster TEXT	A cluster name. If provided, either reuse an existing cluster with that name or provision a new cluster with that name. Otherwise provision a new cluster with an autogenerated name.
    --dryrun	If True, do not actually run the job.
    -s, --detach-setup	If True, run setup in non-interactive mode as part of the job itself. You can safely ctrl-c to detach from logging, and it will not interrupt the setup process. To see the logs again after detaching, use sky logs. To cancel setup, cancel the job via sky cancel. Useful for long-running setup commands.
    -d, --detach-run	If True, as soon as a job is submitted, return from this call and do not stream execution logs.
    --docker	If used, runs locally inside a docker container.
    -n, --name TEXT	Task name. Overrides the ‘name’ config in the YAML if both are supplied.
    --workdir DIRECTORY	If specified, sync this dir to the remote working directory, where the task will be invoked. Overrides the ‘workdir’ config in the YAML if both are supplied.
    --cloud TEXT	The cloud to use. If specified, overrides the ‘resources.cloud’ config. Passing ‘none’ resets the config.
    --region TEXT	The region to use. If specified, overrides the ‘resources.region’ config. Passing ‘none’ resets the config.
    --zone TEXT	The zone to use. If specified, overrides the ‘resources.zone’ config. Passing ‘none’ resets the config.
    --num-nodes INTEGER	Number of nodes to execute the task on. Overrides the ‘num_nodes’ config in the YAML if both are supplied.
    --cpus TEXT	Number of vCPUs each instance must have (e.g., --cpus=4 (exactly 4) or --cpus=4+ (at least 4)). This is used to automatically select the instance type.
    --memory TEXT	Amount of memory each instance must have (e.g., --memory=4 (exactly 4 GB) or --memory=4+ (at least 4 GB)). This is used to automatically select the instance type.
    --hcpu INTEGER	Head node CPU. Overrides the ‘resources.head_cpu’ config in the YAML if both are supplied.
    --hmemory INTEGER	Head node memory in GB. Overrides the ‘resources.head_memory’ config in the YAML if both are supplied.
    --htype TEXT	Head node instance type. Overrides the ‘resources.head_instance_type’ config in the YAML if both are supplied.
    --disk-size INTEGER	OS disk size in GBs.
    --disk-tier [low|medium|high|best|none]	OS disk tier. Could be one of low, medium, high, best, none. If best is specified, use the best possible disk tier. If none is specified, enforce to use default value and override the option in task YAML. Default: medium
    --use-spot / --no-use-spot	Whether to request spot instances. If specified, overrides the ‘resources.use_spot’ config.
    --image-id TEXT	Custom image id for launching the instances. Passing ‘none’ resets the config.
    --env-file DOTENV_VALUES	Path to a dotenv file with environment variables to set on the remote node.
    --env _PARSE_ENV_VAR	Environment variable to set on the remote node. It can be specified multiple times. Examples: 1. --env MY_ENV=1: set $MY_ENV on the cluster to be 1. 2. --env MY_ENV2=$HOME: set $MY_ENV2 on the cluster to be the same value of $HOME in the local environment where the CLI command is run. 3. --env MY_ENV3: set $MY_ENV3 on the cluster to be the same value of $MY_ENV3 in the local environment.
    --gpus TEXT	Type and number of GPUs to use. Example values: ‘V100:8’, ‘V100’ (short for a count of 1), or ‘V100:0.5’ (fractional counts are supported by the scheduling framework). If a new cluster is being launched by this command, this is the resources to provision. If an existing cluster is being reused, this is seen as the task demand, which must fit the cluster’s total resources and is used for scheduling the task. Overrides the ‘accelerators’ config in the YAML if both are supplied. Passing ‘none’ resets the config.
    -t, --instance-type TEXT	The instance type to use. If specified, overrides the ‘resources.instance_type’ config. Passing ‘none’ resets the config.
    --ports TEXT	Ports to open on the cluster. If specified, overrides the ‘ports’ config in the YAML.
    -i, --idle-minutes-to-autostop INTEGER	Automatically stop the cluster after this many minutes of idleness, i.e., no running or pending jobs in the cluster’s job queue. Idleness gets reset whenever setting-up/running/pending jobs are found in the job queue. Setting this flag is equivalent to running sky launch -d ... and then sky autostop -i <minutes>. If not set, the cluster will not be autostopped.
    --down	Autodown the cluster: tear down the cluster after all jobs finish (successfully or abnormally). If –idle-minutes-to-autostop is also set, the cluster will be torn down after the specified idle time. Note that if errors occur during provisioning/data syncing/setting up, the cluster will not be torn down for debugging purposes.
    -r, --retry-until-up	Whether to retry provisioning infinitely until the cluster is up, if we fail to launch the cluster on any possible region/cloud due to unavailability errors.
    -y, --yes	Skip confirmation prompt.
    --no-setup	Skip setup phase when (re-)launching cluster.
    --clone-disk-from, --clone TEXT	[Experimental] Clone disk from an existing cluster to launch a new one. This is useful when the new cluster needs to have the same data on the boot disk as an existing cluster.
    -h, --help	Show help for the command..

d3x umount

• Usage

  CommandExample

  d3x umount [OPTIONS] MOUNTPATH
  

• Function

  Unmount bucket previously mounted using d3x mount command.

• Options

  Option	Description
  --help	Show help for the command..

d3x version

• Usage

  CommandExample

  d3x version [OPTIONS]
  

• Function

  Show DKubeX version.

• Options

  Option	Description
  -o, --output [yaml|json]	supported only json,yaml
  --help	Show help for the command..

d3x ws

• Usage

  CommandExample

  d3x ws [OPTIONS] COMMAND [ARGS]...
  

• Function

  Commands related to DKubeX workspace.

• Options

  Option	Description
  --help	Show this message and exit.

• Commands

  Command	Description
  code	Start local VSCode and connect to D3X workspace
  list	List d3x workspaces
  optuna	Group for optuna commands.
  port-forward	Forward port from workspace to local computer
  reset	reset given workspace to default image
  scp	COPY files(s)/directory from/to D3X workspace
  ssh	SSH to D3X workspace
  ssh-copy-id	Copy ssh public key to D3X user’s home
  start	Start given workspace
  start-app	Start given workspace app
  stop	Stop given workspace
  stop-app	Stop given workspace App
  tensorboard	Group for tensorboard commands.
  vnc	VNC connection to D3X workspace

  d3x ws coded3x ws listd3x ws optunad3x ws port-forwardd3x ws resetd3x ws scpd3x ws sshd3x ws ssh-copy-idd3x ws startd3x ws start-appd3x ws stopd3x ws stop-appd3x ws tensorboardd3x ws vnc

  • Usage

    CommandExample

    d3x ws code [OPTIONS] WORKSPACE_NAME
    

  • Function

    Start local VSCode and connect to D3X workspace.

  • Options

    Option	Description
    --help	Show help for the command..