d3x Commands

d3x commands are used for managing the user workspace, apikeys, ray clusters, user apps and deployments. This page provides information about all of the d3x commands that are currently available, along with their functions, usage syntax and available options.

Usage

d3x [OPTIONS] COMMAND [ARGS]...

Function

Group for d3x commands.

Options

Option

Description

-d, --debug

enable requests debugging.

--profile TEXT

cluster profile

--help

Show help for the command..

Commands

Command

Description

about

Details about d3x CLI configuration.

apikey

Commands to generate and get DKubeX API keys.

apps

Commands regarding creating and managing user-defined applications.

backup

Commands for backing up datasets.

code

Start local VSCode and connect to DKubeX workspace.

dataset

Commands for creating, managing and using datasets.

fm

Commands regarding using foundation models.

list

List DKubeX workspaces.

llms

Commands regarding using LLMs.

login

Login to DKubeX with particular user profile in the CLI.

mlflow

Commands for using and managing experiments and models in MLFlow from CLI.

models

List & Manage MLFlow models registry.

mount

Mount S3 bucket in read/write mode.

port-forward

Forward port from workspace to local computer.

profile

Commands regarding user profiles in DKubeX.

ray

Commands regarding Ray jobs and Ray clusters.

reset

Reset given workspace to default image

restore

Commands for restoring databases.

scp

Copy files(s)/directory from/to DKubeX workspace.

serve

Commands for creating model deployments.

sky

Commands for managing SkyPilot clusters, resources and jobs.

ssh

SSH to DKubeX workspace.

ssh-copy-id

Copy ssh public key to DKubeX user’s home.

start

Start given workspace.

start-app

Start given workspace app.

stop

Stop given workspace.

stop-app

Stop given workspace app.

tensorboard

Commands regarding using Tensorboard.

top

Show workspace resource usage.

umount

Unmount bucket previously mounted using d3x mount command.

version

Show DKubeX version.

vnc

VNC connection to DKubeX workspace.

d3x about

  • Usage

    d3x about [OPTIONS]
    
  • Function

    Details about d3x CLI config

  • Options

    Option

    Description

    -o, --output [yaml|json]

    output format (supported- json, yaml)

    --help

    Show help for the command..

d3x apikey

  • Usage

    d3x apikey [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands to generate and get DKubeX API keys.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    generate

    Generate new user API key

    get

    Get current user API key

    • Usage

      d3x apikey generate [OPTIONS]
      
    • Function

      Generate new user API key.

    • Options

      Option

      Description

      -o, --output [yaml|json]

      output format (supported- json, yaml)

      --help

      Show help for the command..

d3x apps

  • Usage

    d3x apps [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding creating and managing user-defined applications.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    create

    Launch user-defined application on DKubeX

    delete

    Delete user-defined application

    list

    List user-defined applications

    open

    Open user-defined application in browser

    • Usage

      d3x apps create [OPTIONS] [COMMAND]...
      
    • Function

      Launch user-defined application on DKubeX.

    • Options

      Option

      Description

      -c, --config TEXT

      Path to the YAML configuration file [required]

      --help

      Show help for the command..

d3x backup

  • Usage

    d3x backup [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for backing up datasets.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    weaviate

    command for weaviate backup

    • Usage

      d3x backup weaviate [OPTIONS]
      
    • Function

      Command for dataset backup to Weaviate.

    • Options

      Option

      Description

      -d, --dataset TEXT

      A name to represent the ingested docs [required]

      -i, --id TEXT

      An ID for the backup [required]

      -b, --backend [filesystem|s3]

      s3 or filesystem to save the backup [required]

      --help

      Show help for the command..

d3x code

  • Usage

    d3x code [OPTIONS] WORKSPACE_NAME
    
  • Function

    Start local VSCode and connect to DKubeX workspace

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x dataset

  • Usage

    d3x dataset [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for creating, managing and using datasets.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    delete

    Deletes specified dataset if present from the database

    evaluate

    Run evaluation pipeline over the given dataset.

    ingest

    Ingest the documents from provided source into the database.

    list

    lists all the datasets present in database

    query

    Query over documents using the selected pipeline.

    show

    shows documents present in the specified dataset

    • Usage

      d3x dataset delete [OPTIONS] DATASET
      
    • Function

      Deletes specified dataset if present from the database

    • Options

      Option

      Description

      --help

      Show help for the command..

d3x fm

  • Usage

    d3x fm [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding using foundation models.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    delete

    Delete FM job.

    get

    Get FM job details.

    jobs

    List FM jobs.

    logs

    Show logs of given FM job.

    trainchunks

    Create train and/or test chunks for finetuning.

    tune

    LLM Finetuning process related commands.

    • Usage

      d3x fm delete [OPTIONS] JOB_NAME
      
    • Function

      Delete FM job.

    • Options

      Option

      Description

      --help

      Show help for the command..

d3x list

  • Usage

    d3x list [OPTIONS]
    
  • Function

    List d3x workspaces

  • Options

    Option

    Description

    -o, --output [yaml|json]

    supported only json,yaml

    --help

    Show help for the command..

d3x llms

  • Usage

    d3x llms [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding using LLMs.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    deploy

    Deploy a LLM on dkubex.

    list

    List the supported LLMs

    • Usage

      d3x llms deploy [OPTIONS]
      
    • Function

      Deploy a LLM on dkubex.

    • Options

      Option

      Description

      -n, --name TEXT

      name of the deployment [required]

      -m, --model TEXT

      name or path of the model to deploy for serving

      --base_model TEXT

      base model name

      --token TEXT

      hugging face token

      --config TEXT

      github raw content url or local config path

      --mlflow TEXT

      name of the mlflow registered model along with version (example: diffusion:1)

      --image TEXT

      customized docker image for llm deployment

      --type TEXT

      instance type

      --min_replicas TEXT

      minimum replicas

      --max_replicas TEXT

      maximum replicas

      --publish

      publish the deployment

      -o, --output [yaml|json]

      supported only json,yaml

      -sky, --remote-sky

      If the model should be deployed on remote sky cluster.

      --sky-accelerator TEXT

      accelerator for remote sky cluster

      -k, --dkubex-apikey TEXT

      API key for remote clusters to reach dkubex.

      --sky-yaml TEXT

      Path to custom yaml file for sky serve.

      --help

      Show help for the command..

d3x login

  • Usage

    d3x login [OPTIONS]
    
  • Function

    Login to DKubeX with particular user profile in the CLI.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x mlflow

  • Usage

    d3x mlflow [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for using and managing experiments and models in MLFlow from CLI.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    experiments

    Commands for managing experiments in MLFlow.

    models

    Commands for managing models in MLFlow.

    • Usage

      d3x mlflow experiments [OPTIONS] COMMAND [ARGS]...
      
    • Function

      Commands for managing experiments in MLFlow.

    • Options

      Option

      Description

      --help

      Show help for the command..

    • Commands

      Command

      Description

      archive

      Archive experiment in MLFlow

      delete

      Delete experiment in MLFlow

      list

      List experiments in MLFlow

      restore

      Restore experiment in MLFlow

      • Usage

        d3x mlflow experiments archive [OPTIONS] ID
        
      • Function

        Archive experiment in MLFlow.

      • Options

        Option

        Description

        --help

        Show help for the command..

d3x models

  • Usage

    d3x models [OPTIONS] COMMAND [ARGS]...
    
  • Function

    List & Manage MLFlow models registry.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    import

    Import saved model to MLFlow.

    list

    List models registered in MLFlow.

    • Usage

      d3x models import [OPTIONS] NAME {pytorch|tensorflow|sklearn|xgboost|custom_model} SAVED_MODEL_PATH
      
    • Function

      Import saved model to MLFlow.

    • Options

      Option

      Description

      --class_path TEXT

      pytorch class file path

      --class_instance TEXT

      class instance

      -o, --output [yaml|json]

      supported only json,yaml

      --help

      Show help for the command..

d3x mount

  • Usage

    d3x mount [OPTIONS] [BUCKET] [MOUNTPATH]
    
  • Function

    Mount S3 bucket in read/write mode.

  • Options

    Option

    Description

    -a, --all

    Mounts all the previous buckets.

    -d, --dry-run

    Runs a dry run of mounting bucket, does not mount the buckets.

    --help

    Show help for the command..

d3x port-forward

  • Usage

    d3x port-forward [OPTIONS] WORKSPACE_NAME SERVICE TARGET_PORT LOCAL_PORT
    
  • Function

    Forward port from workspace to local computer.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x profile

  • Usage

    d3x profile [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding user profiles in DKubeX.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    configure

    Configure user profile.

    delete

    Delete user profile.

    list

    List user profiles.

    set

    Set default user profile.

    • Usage

      d3x profile configure [OPTIONS]
      
    • Function

      Configure user profile.

      • Options

        Option

        Description

        --help

        Show help for the command..

d3x ray

  • Usage

    d3x ray [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding Ray jobs and Ray clusters.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    activate

    Set Ray cluster as active cluster.

    address

    List Ray clusters along with their cluster Ray address.

    create

    Create Ray cluster.

    dashboard

    Open Ray dashboard.

    delete

    Delete Ray cluster.

    get

    Get a Ray cluster.

    job

    Commands regarding managing Ray jobs in currently active Ray cluster.

    list

    List Ray clusters.

    • Usage

      d3x ray activate [OPTIONS] CLUSTER
      
    • Function

      Set Ray cluster as active cluster.

    • Options

      Option

      Description

      -o, --output [yaml|json]

      supported only json,yaml

      --help

      Show help for the command..

d3x reset

  • Usage

    d3x reset [OPTIONS]
    
  • Function

    Reset given workspace to default image.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x restore

  • Usage

    d3x restore [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for restoring databases.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    weaviate

    Command for Weaviate restore.

    • Usage

      d3x restore weaviate [OPTIONS]
      
    • Function

      Command for Weaviate restore.

    • Options

      Option

      Description

      -i, --id TEXT

      An ID for the backup [required]

      -b, --backend [filesystem|s3]

      s3 or filesystem to save the backup [required]

      --help

      Show help for the command..

d3x scp

  • Usage

    d3x scp [OPTIONS] SOURCE TARGET
    
  • Function

    Copy files(s)/directory from/to DKubeX workspace.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x serve

  • Usage

    d3x serve [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for creating model deployments.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    create

    Create deployment

    delete

    Delete deployment

    get

    Get Deployment

    list

    List Deployments

    status

    Check deployment Status

    • Usage

      d3x serve create [OPTIONS]
      
    • Function

      Create deployment.

    • Options

      Option

      Description

      -n, --name TEXT

      name of the ray serve [required]

      -r, --registry_type [hugging_face|mlflow|local|none]

      registry type [required]

      --hface_repoid TEXT

      hugging face repo id

      --hface_tokenizer TEXT

      hugging face tokenizer

      --hface_classifier TEXT

      hugging face classifier

      --hface_token TEXT

      hugging face token

      --model TEXT

      mlflow model name

      --model_version TEXT

      mlflow model version

      --depfilepath TEXT

      deployment filepath

      --working_dir TEXT

      local serving directory path

      --is_private_repo

      private repo

      --repo_name TEXT

      github repo name

      --repo_org TEXT

      github personal/organization account

      --access_token TEXT

      github access token

      --branch_name TEXT

      github branch name

      --commit_id TEXT

      github commit id

      --unprotected

      serving without token

      -e, --env TEXT

      environment variable

      --ngpus TEXT

      number of gpus

      --min_cpu TEXT

      min number of cpus for cluster creation

      --max_cpu TEXT

      max number of cpus per cluster creation

      --min_memory TEXT

      min memory for cluster cluster creation

      --max_memory TEXT

      max memory for cluster cluster creation

      --min_replicas TEXT

      min deployment replicas

      --max_replicas TEXT

      max deployment replicas

      --max_concurrent_requests TEXT

      max concurrent requests

      --description TEXT

      deployment description

      --image TEXT

      cluster image

      --publish

      publish the deployment

      --no_rsrc_limits

      no resource limits

      --type TEXT

      node selector

      --timeout INTEGER

      deployment timeout

      --kserve

      enable kserve deployment

      --model_format [tensorflow|sklearn|pytorch|xgboost|pmml|lightgbm|paddle|mlflow|custom]

      model format in case of kserve deployment

      --model_path TEXT

      local path of model for kserve deployment

      --protocol_version [v1|v2]

      dataplane protocol

      -o, --output [yaml|json]

      supported only json,yaml

      --help

      Show help for the command..

d3x sky

  • Usage

    d3x sky [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands for managing SkyPilot clusters, resources and jobs.

  • Options

    Option

    Description

    --install-shell-completion [bash|zsh|fish|auto]

    Install shell completion for the specified shell.

    --uninstall-shell-completion [bash|zsh|fish|auto]

    Uninstall shell completion for the specified shell.

    -v, --version

    Show the version and exit.

    -c, --commit

    Show the commit hash and exit

    -h, --help

    Show help for the command..

  • Commands

    Command

    Description

    launch

    Launch a cluster or task.

    exec

    Execute a task or command on an existing cluster.

    status

    Show clusters.

    cost-report

    Show estimated costs for launched clusters.

    queue

    Show the job queue for cluster(s).

    logs

    Tail the log of a job.

    cancel

    Cancel job(s).

    stop

    Stop cluster(s).

    autostop

    Schedule an autostop or autodown for cluster(s).

    start

    Restart cluster(s).

    down

    Tear down cluster(s).

    check

    Check which clouds are available to use.

    show-gpus

    Show supported GPU/TPU/accelerators and their prices.

    storage

    Commands for using SkyPilot Storage CLI.

    bench

    Commands for usingSkyPilot Benchmark CLI.

    spot

    Commands for using managed Spot CLI (spot instances with auto-recovery).

    serve

    Commands for using SkyServe CLI (multi-region, multi-cloud serving).

    • Usage

      d3x sky launch [OPTIONS] [ENTRYPOINT]...
      
    • Function

      Launch a cluster or task. If ENTRYPOINT points to a valid YAML file, it is read in as the task specification. Otherwise, it is interpreted as a bash command. In both cases, the commands are run under the task’s workdir (if specified) and they undergo job queue scheduling.

    • Options

      Option

      Description

      -c, --cluster TEXT

      A cluster name. If provided, either reuse an existing cluster with that name or provision a new cluster with that name. Otherwise provision a new cluster with an autogenerated name.

      --dryrun

      If True, do not actually run the job.

      -s, --detach-setup

      If True, run setup in non-interactive mode as part of the job itself. You can safely ctrl-c to detach from logging, and it will not interrupt the setup process. To see the logs again after detaching, use sky logs. To cancel setup, cancel the job via sky cancel. Useful for long-running setup commands.

      -d, --detach-run

      If True, as soon as a job is submitted, return from this call and do not stream execution logs.

      --docker

      If used, runs locally inside a docker container.

      -n, --name TEXT

      Task name. Overrides the ‘name’ config in the YAML if both are supplied.

      --workdir DIRECTORY

      If specified, sync this dir to the remote working directory, where the task will be invoked. Overrides the ‘workdir’ config in the YAML if both are supplied.

      --cloud TEXT

      The cloud to use. If specified, overrides the ‘resources.cloud’ config. Passing ‘none’ resets the config.

      --region TEXT

      The region to use. If specified, overrides the ‘resources.region’ config. Passing ‘none’ resets the config.

      --zone TEXT

      The zone to use. If specified, overrides the ‘resources.zone’ config. Passing ‘none’ resets the config.

      --num-nodes INTEGER

      Number of nodes to execute the task on. Overrides the ‘num_nodes’ config in the YAML if both are supplied.

      --cpus TEXT

      Number of vCPUs each instance must have (e.g., --cpus=4 (exactly 4) or --cpus=4+ (at least 4)). This is used to automatically select the instance type.

      --memory TEXT

      Amount of memory each instance must have (e.g., --memory=4 (exactly 4 GB) or --memory=4+ (at least 4 GB)). This is used to automatically select the instance type.

      --hcpu INTEGER

      Head node CPU. Overrides the ‘resources.head_cpu’ config in the YAML if both are supplied.

      --hmemory INTEGER

      Head node memory in GB. Overrides the ‘resources.head_memory’ config in the YAML if both are supplied.

      --htype TEXT

      Head node instance type. Overrides the ‘resources.head_instance_type’ config in the YAML if both are supplied.

      --disk-size INTEGER

      OS disk size in GBs.

      --disk-tier [low|medium|high|best|none]

      OS disk tier. Could be one of low, medium, high, best, none. If best is specified, use the best possible disk tier. If none is specified, enforce to use default value and override the option in task YAML. Default: medium

      --use-spot / --no-use-spot

      Whether to request spot instances. If specified, overrides the ‘resources.use_spot’ config.

      --image-id TEXT

      Custom image id for launching the instances. Passing ‘none’ resets the config.

      --env-file DOTENV_VALUES

      Path to a dotenv file with environment variables to set on the remote node.

      --env _PARSE_ENV_VAR

      Environment variable to set on the remote node. It can be specified multiple times. Examples: 1. --env MY_ENV=1: set $MY_ENV on the cluster to be 1. 2. --env MY_ENV2=$HOME: set $MY_ENV2 on the cluster to be the same value of $HOME in the local environment where the CLI command is run. 3. --env MY_ENV3: set $MY_ENV3 on the cluster to be the same value of $MY_ENV3 in the local environment.

      --gpus TEXT

      Type and number of GPUs to use. Example values: ‘V100:8’, ‘V100’ (short for a count of 1), or ‘V100:0.5’ (fractional counts are supported by the scheduling framework). If a new cluster is being launched by this command, this is the resources to provision. If an existing cluster is being reused, this is seen as the task demand, which must fit the cluster’s total resources and is used for scheduling the task. Overrides the ‘accelerators’ config in the YAML if both are supplied. Passing ‘none’ resets the config.

      -t, --instance-type TEXT

      The instance type to use. If specified, overrides the ‘resources.instance_type’ config. Passing ‘none’ resets the config.

      --ports TEXT

      Ports to open on the cluster. If specified, overrides the ‘ports’ config in the YAML.

      -i, --idle-minutes-to-autostop INTEGER

      Automatically stop the cluster after this many minutes of idleness, i.e., no running or pending jobs in the cluster’s job queue. Idleness gets reset whenever setting-up/running/pending jobs are found in the job queue. Setting this flag is equivalent to running sky launch -d ... and then sky autostop -i <minutes>. If not set, the cluster will not be autostopped.

      --down

      Autodown the cluster: tear down the cluster after all jobs finish (successfully or abnormally). If –idle-minutes-to-autostop is also set, the cluster will be torn down after the specified idle time. Note that if errors occur during provisioning/data syncing/setting up, the cluster will not be torn down for debugging purposes.

      -r, --retry-until-up

      Whether to retry provisioning infinitely until the cluster is up, if we fail to launch the cluster on any possible region/cloud due to unavailability errors.

      -y, --yes

      Skip confirmation prompt.

      --no-setup

      Skip setup phase when (re-)launching cluster.

      --clone-disk-from, --clone TEXT

      [Experimental] Clone disk from an existing cluster to launch a new one. This is useful when the new cluster needs to have the same data on the boot disk as an existing cluster.

      -h, --help

      Show help for the command..

d3x ssh

  • Usage

    d3x ssh [OPTIONS] [COMMAND]...
    
  • Function

    SSH to DKubeX workspace.

  • Options

    Option

    Description

    -w TEXT

    workspace name

    --help

    Show help for the command..

d3x ssh-copy-id

  • Usage

    d3x ssh-copy-id [OPTIONS] PUBLIC_KEY
    
  • Function

    Copy ssh public key to DKubeX user’s home.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x start

  • Usage

    d3x start [OPTIONS]
    
  • Function

    Start given workspace.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x start-app

  • Usage

    d3x start-app [OPTIONS] APP_NAME
    
  • Function

    Start given workspace app.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x stop

  • Usage

    d3x stop [OPTIONS]
    
  • Function

    Stop given workspace.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x stop-app

  • Usage

    d3x stop-app [OPTIONS] APP_NAME
    
  • Function

    Stop given workspace App.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x tensorboard

  • Usage

    d3x tensorboard [OPTIONS] COMMAND [ARGS]...
    
  • Function

    Commands regarding using Tensorboard.

  • Options

    Option

    Description

    --help

    Show help for the command..

  • Commands

    Command

    Description

    get-logdir

    Getting log directory for tensorboard.

    set-logdir

    Setting log directory for tensorboard app inside workspace.

    • Usage

      d3x tensorboard get-logdir [OPTIONS]
      
    • Function

      Getting log directory for tensorboard.

      • Options

        Option

        Description

        -o, --output [yaml|json]

        supported only json,yaml

        --help

        Show help for the command..

d3x top

  • Usage

    d3x top [OPTIONS] [WORKSPACE]
    
  • Function

    Show workspace resource usage.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x umount

  • Usage

    d3x umount [OPTIONS] MOUNTPATH
    
  • Function

    Unmount bucket previously mounted using d3x mount command.

  • Options

    Option

    Description

    --help

    Show help for the command..

d3x version

  • Usage

    d3x version [OPTIONS]
    
  • Function

    Show DKubeX version.

  • Options

    Option

    Description

    -o, --output [yaml|json]

    supported only json,yaml

    --help

    Show help for the command..

d3x vnc

  • Usage

    d3x vnc [OPTIONS] WORKSPACE_NAME
    
  • Function

    VNC connection to DKubeX workspace.

  • Options

    Option

    Description

    --help

    Show help for the command..