> ## Documentation Index
> Fetch the complete documentation index at: https://docs.phala.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Advanced CVM Settings

> Init scripts, ephemeral data disks, annotations, other runners and raw app-compose input for CVMs created or updated through the API.

These settings are set through the API, on the create request (`POST /cvms`) or with [`PATCH /cvms/{cvm_id}`](https://docs.phala.com/phala-cloud/update/update-cvm-configuration), or with the matching [`phala deploy`](https://docs.phala.com/phala-cloud/phala-cloud-cli/deploy) flags. Some of them sit behind a feature flag and are available on request: [contact support](https://docs.phala.com/phala-cloud/support.md) to turn them on for your workspace.

## Init scripts

Init scripts are small scripts Phala Cloud provides. The CVM runs them on every boot, before your containers start. You pick them by name:

| Name | What it does | Needs |
| - | - | - |
| `docker_cleanup` | Erases every local container store on each boot: images, volumes and all containerd namespaces. Each boot pulls images again and starts with empty Docker volumes. | dstack OS 0.6.x, `ext4` storage, the `docker-compose` runner |
| `env_merge` | On each boot, merges the encrypted environment variables you sent into the ones the CVM stored before, limited to `allowed_envs`. | dstack OS 0.6.0 or newer. Available on request. |

The scripts always run in this order: `docker_cleanup`, then `env_merge`.

### How do I choose init scripts?

Send the full list. It replaces the scripts the CVM runs now, and `[]` removes them all:

```json theme={"system"}
{ "init_scripts": ["docker_cleanup", "env_merge"] }
```

Use the same field on create and on `PATCH /cvms/{cvm_id}`. The create request still accepts the older forms `init_scripts: {"env_merge": true}` and `enabled_init_scripts_docker_cleanup`.

Storage is fixed when you create a CVM, and the default is `zfs`. To use `docker_cleanup`, create the CVM with `ext4` storage.

### Why does turning on docker\_cleanup ask for confirmation?

Because the next boot erases your images and Docker volumes, and every boot after that does it again. On an existing CVM, the update fails with `ERR-03-048` unless you also send `"confirm_data_wipe": true`. Back up any data in Docker volumes before you turn it on.

### Does each CVM have its own init scripts?

Yes. Init scripts are part of each CVM's compose. Changing them is a compose change: the CVM gets a new compose hash and restarts, and on Onchain KMS the new hash must be registered. Redeploying an older revision brings back that revision's init scripts.

To see what a CVM runs and what you could pick, call `GET /cvms/{cvm_id}/init-scripts`. Each item says whether the CVM runs the script (`selected`), whether you can pick it (`selectable`) and, if not, why (`reason`).

## Ephemeral data disk

With `requirements.ephemeral_data_disk` set to `true`, the CVM starts every boot with an empty data disk. The guest formats the disk before any app code runs, so nothing from earlier boots is left. Use it for workloads that must not keep data between boots.

It needs dstack OS 0.6.3 or newer.

To turn it on at create, put it in `compose_file.requirements`. To turn it on for an existing CVM, confirm the wipe:

```json theme={"system"}
{
  "compose": { "requirements": { "ephemeral_data_disk": true } },
  "confirm_data_wipe": true
}
```

Without `confirm_data_wipe`, the update fails with `ERR-03-048`. Turning it off needs no confirmation.

## Annotations

Annotations sit behind a feature flag and are available on request. Without it, every annotations request, and a create request that carries annotations, fails with `403` and `ERR-03-049`.

Annotations are key/value labels for tools on the host, for example a wrapper that reads the VM's settings to place its memory or pin its vCPUs. The node keeps them with the VM.

They are host configuration, not part of your app. They never reach the CVM, are not measured and are not part of the compose hash, so attestation does not cover them. Do not rely on them for security.

### How do I set annotations?

* At create: add an `annotations` map to the create request.
* On an existing CVM: `PUT /cvms/{cvm_id}/annotations` with `{"annotations": {"memory_tier": "fast"}}`. This replaces all annotations. To clear them, send `{"annotations": {}}`.
* To read them: `GET /cvms/{cvm_id}/annotations`.
* From the CLI: [`phala annotations`](https://docs.phala.com/phala-cloud/phala-cloud-cli/annotations) shows them, and `phala annotations <cvm-id> memory_tier=fast` sets a key and keeps the others.

Keys must not be empty. The node must run dstack VMM 0.6.1 or newer, or the request fails with `ERR-03-050`.

Setting annotations does not restart the CVM. Some host tools read them only when the VM starts. For those, restart the CVM after you change them.

### Do annotations follow the CVM?

Phala Cloud does not keep annotations as part of the CVM's state and never restores them. The node holds them. The audit record of a create request may include the annotations it carried.

* A replica starts without annotations. Set them again on the replica.
* A CVM moved to another node keeps whatever its VM settings on the node carry.
* Other updates to the CVM never change them.

## Other runners

The runner is the program that starts your app inside the CVM. The default is `docker-compose`, and it is the only runner open to everyone. These two sit behind a feature flag and are available on request. Without it, a create request that picks one fails with `ERR-03-045`.

* `nerdctl-compose` runs your Docker Compose YAML with nerdctl. It needs dstack OS 0.6.0 or newer. It cannot build images, so the YAML must not have `build:` sections. It is the only runner that accepts `snapshotter`.
* `bash` runs the script in `bash_script` instead of Docker Compose YAML. There are no containers, so the CVM has no container list or container logs, and `docker_cleanup` is not available. If you turn on `requirements.health_check`, you must also set `requirements.health_status_file`. To change the script later, send `{"compose": {"bash_script": "..."}}` with `PATCH /cvms/{cvm_id}`.

Set the runner in `compose_file.runner` when you create the CVM. It cannot change afterwards.

### Why does my bash CVM stay in processing?

Because your script has not exited. The CVM runs `bash_script` in the foreground and finishes booting only after the script returns. Until then, the CVM stays in `processing` with the boot step "running main script", and it never reaches `running`. A script such as `while true; do sleep 300; done` keeps the CVM booting forever.

Start long-running work in the background and let the script return:

```bash theme={"system"}
(my-server --port 8080) >/dev/null 2>&1 &
```

A bash CVM has no containers, so `phala ps` lists none and there are no container logs. The script's output is in the CVM's service journal.

## Raw app-compose input

Raw app-compose input sits behind the same feature flag as the other runners. Without it, the create request fails with `ERR-03-045`.

Instead of `compose_file`, a create request can send `compose_file_raw`: a whole app-compose JSON file as one string. Send one or the other, not both.

Phala Cloud reads the file with the same rules as `compose_file` and rejects fields it does not know. It then writes the compose the same way it writes any other. The bytes the CVM runs, and the compose hash, come from that output, not from the bytes you sent.

After create, the CVM is like any other. Update it with [`PATCH /cvms/{cvm_id}`](https://docs.phala.com/phala-cloud/update/update-cvm-configuration).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.