Configuration Files

Note

Please note that XML-based Settings Profiles and configuration files are currently not supported for ClickHouse Cloud. Therefore, in ClickHouse Cloud, you won't find a config.xml file. Instead, you should use SQL commands to manage settings through Settings Profiles.

For further details, see "Configuring Settings"

The ClickHouse server can be configured with configuration files in XML or YAML syntax. In most installation types, the ClickHouse server runs with /etc/clickhouse-server/config.xml as default configuration file, but it is also possible to specify the location of the configuration file manually at server startup using command line option --config-file or -C. Additional configuration files may be placed into directory config.d/ relative to the main configuration file, for example into directory /etc/clickhouse-server/config.d/. Files in this directory and the main configuration are merged in a preprocessing step before the configuration is applied in ClickHouse server. Configuration files are merged in alphabetical order. To simplify updates and improve modularization, it is best practice to keep the default config.xml file unmodified and place additional customization into config.d/. The ClickHouse keeper configuration lives in /etc/clickhouse-keeper/keeper_config.xml. Thus, additional files need to be placed in /etc/clickhouse-keeper/keeper_config.d/.

It is possible to mix XML and YAML configuration files, for example you could have a main configuration file config.xml and additional configuration files config.d/network.xml, config.d/timezone.yaml and config.d/keeper.yaml. Mixing XML and YAML within a single configuration file is not supported. XML configuration files should use <clickhouse>...</clickhouse> as top-level tag. In YAML configuration files, clickhouse: is optional, if absent the parser inserts it automatically.

Merging Configuration

Two configuration files (usually the main configuration file and another configuration files from config.d/) are merged as follows:

• If a node (i.e. a path leading to an element) appears in both files and does not have attributes replace or remove, it is included in the merged configuration file and children from both nodes are included and merged recursively.
• If one of both nodes contains attribute replace, it is included in the merged configuration file but only children from the node with attribute replace are included.
• If one of both nodes contains attribute remove, the node is not included in the merged configuration file (if it exists already, it is deleted).

Example:

and

generates merged configuration file:

Substitution by Environment Variables and ZooKeeper Nodes

To specify that a value of an element should be replaced by the value of an environment variable, you can use attribute from_env.

Example with $MAX_QUERY_SIZE = 150000:

which is equal to

The same is possible using from_zk (ZooKeeper node):

which is equal to

Default Values

An element with from_env or from_zk attribute may additionally have attribute replace="1" (the latter must appear before from_env/from_zk). In this case, the element may define a default value. The element takes on the value of the environment variable or ZooKeeper node if set, otherwise the default value.

Previous example but assuming MAX_QUERY_SIZE is not set:

Result:

Substitution with File Content

It is also possible to replace parts of the configuration by file contents. This can be done in two ways:

• Substituting Values: If an element has attribute incl, its value will be replaced by the content of the referenced file. By default, the path to the file with substitutions is /etc/metrika.xml. This can be changed in the include_from element in the server config. The substitution values are specified in /clickhouse/substitution_name elements in this file. If a substitution specified in incl does not exist, it is recorded in the log. To prevent ClickHouse from logging missing substitutions, specify attribute optional="true" (for example, settings for macros).

• Substituting elements: If you want to replace the entire element with a substitution, use include as the element name. Element name include can be combined with attribute from_zk = "/path/to/node". In this case, the element value is replaced by the contents of the ZooKeeper node at /path/to/node. This also works with you store an entire XML subtree as a Zookeeper node, it will be fully inserted into the source element.

Example:

If you want to merge the substituting content with the existing configuration instead of appending you can use attribute merge="true", for example: <include from_zk="/some_path" merge="true">. In this case, the existing configuration will be merged with the content from the substitution and the existing configuration settings will be replaced with values from substitution.

Encrypting and Hiding Configuration

You can use symmetric encryption to encrypt a configuration element, for example, a plaintext password or private key. To do so, first configure the encryption codec, then add attribute encrypted_by with the name of the encryption codec as value to the element to encrypt.

Unlike attributes from_zk, from_env and incl, or element include, no substitution (i.e. decryption of the encrypted value) is performed in the preprocessed file. Decryption happens only at runtime in the server process.

Example:

To encrypt a value, you can use the (example) program encrypt_decrypt:

Example:

Even with encrypted configuration elements, encrypted elements still appear in the preprocessed configuration file. If this is a problem for your ClickHouse deployment, we suggest two alternatives: Either set file permissions of the preprocessed file to 600 or use attribute hide_in_preprocessed.

Example:

User Settings

The config.xml file can specify a separate config with user settings, profiles, and quotas. The relative path to this config is set in the users_config element. By default, it is users.xml. If users_config is omitted, the user settings, profiles, and quotas are specified directly in config.xml.

Users configuration can be split into separate files similar to config.xml and config.d/. Directory name is defined as users_config setting without .xml postfix concatenated with .d. Directory users.d is used by default, as users_config defaults to users.xml.

Note that configuration files are first merged taking into account settings, and includes are processed after that.

XML example

For example, you can have separate config file for each user like this:

YAML examples

Here you can see default config written in YAML: config.yaml.example.

There are some differences between YAML and XML formats in terms of ClickHouse configurations. Here are some tips for writing a configuration in YAML format.

An XML tag with a text value is represented by a YAML key-value pair

Corresponding XML:

A nested XML node is represented by a YAML map:

Corresponding XML:

To create the same XML tag multiple times, use a YAML sequence:

Corresponding XML:

To provide an XML attribute, you can use an attribute key with a @ prefix. Note that @ is reserved by YAML standard, so must be wrapped in double quotes:

Corresponding XML:

It is also possible to use attributes in YAML sequence:

Corresponding XML:

The aforementioned syntax does not allow to express XML text nodes with XML attributes as YAML. This special case can be achieved using an #text attribute key:

Corresponding XML:

Implementation Details

For each config file, the server also generates file-preprocessed.xml files when starting. These files contain all the completed substitutions and overrides, and they are intended for informational use. If ZooKeeper substitutions were used in the config files but ZooKeeper is not available on the server start, the server loads the configuration from the preprocessed file.

The server tracks changes in config files, as well as files and ZooKeeper nodes that were used when performing substitutions and overrides, and reloads the settings for users and clusters on the fly. This means that you can modify the cluster, users, and their settings without restarting the server.