1# LIBUCL 2 3[![Build Status](https://travis-ci.org/vstakhov/libucl.svg?branch=master)](https://travis-ci.org/vstakhov/libucl) 4 5**Table of Contents** *generated with [DocToc](http://doctoc.herokuapp.com/)* 6 7- [Introduction](#introduction) 8- [Basic structure](#basic-structure) 9- [Improvements to the json notation](#improvements-to-the-json-notation) 10 - [General syntax sugar](#general-syntax-sugar) 11 - [Automatic arrays creation](#automatic-arrays-creation) 12 - [Named keys hierarchy](#named-keys-hierarchy) 13 - [Convenient numbers and booleans](#convenient-numbers-and-booleans) 14- [General improvements](#general-improvements) 15 - [Commments](#commments) 16 - [Macros support](#macros-support) 17 - [Variables support](#variables-support) 18 - [Multiline strings](#multiline-strings) 19- [Emitter](#emitter) 20- [Validation](#validation) 21- [Performance](#performance) 22- [Conclusion](#conclusion) 23 24## Introduction 25 26This document describes the main features and principles of the configuration 27language called `UCL` - universal configuration language. 28 29If you are looking for the libucl API documentation you can find it at [this page](doc/api.md). 30 31## Basic structure 32 33UCL is heavily infused by `nginx` configuration as the example of a convenient configuration 34system. However, UCL is fully compatible with `JSON` format and is able to parse json files. 35For example, you can write the same configuration in the following ways: 36 37* in nginx like: 38 39```nginx 40param = value; 41section { 42 param = value; 43 param1 = value1; 44 flag = true; 45 number = 10k; 46 time = 0.2s; 47 string = "something"; 48 subsection { 49 host = { 50 host = "hostname"; 51 port = 900; 52 } 53 host = { 54 host = "hostname"; 55 port = 901; 56 } 57 } 58} 59``` 60 61* or in JSON: 62 63```json 64{ 65 "param": "value", 66 "param1": "value1", 67 "flag": true, 68 "subsection": { 69 "host": [ 70 { 71 "host": "hostname", 72 "port": 900 73 }, 74 { 75 "host": "hostname", 76 "port": 901 77 } 78 ] 79 } 80} 81``` 82 83## Improvements to the json notation. 84 85There are various things that make ucl configuration more convenient for editing than strict json: 86 87### General syntax sugar 88 89* Braces are not necessary to enclose a top object: it is automatically treated as an object: 90 91```json 92"key": "value" 93``` 94is equal to: 95```json 96{"key": "value"} 97``` 98 99* There is no requirement of quotes for strings and keys, moreover, `:` may be replaced `=` or even be skipped for objects: 100 101```nginx 102key = value; 103section { 104 key = value; 105} 106``` 107is equal to: 108```json 109{ 110 "key": "value", 111 "section": { 112 "key": "value" 113 } 114} 115``` 116 117* No commas mess: you can safely place a comma or semicolon for the last element in an array or an object: 118 119```json 120{ 121 "key1": "value", 122 "key2": "value", 123} 124``` 125### Automatic arrays creation 126 127* Non-unique keys in an object are allowed and are automatically converted to the arrays internally: 128 129```json 130{ 131 "key": "value1", 132 "key": "value2" 133} 134``` 135is converted to: 136```json 137{ 138 "key": ["value1", "value2"] 139} 140``` 141 142### Named keys hierarchy 143 144UCL accepts named keys and organize them into objects hierarchy internally. Here is an example of this process: 145```nginx 146section "blah" { 147 key = value; 148} 149section foo { 150 key = value; 151} 152``` 153 154is converted to the following object: 155 156```nginx 157section { 158 blah { 159 key = value; 160 } 161 foo { 162 key = value; 163 } 164} 165``` 166 167Plain definitions may be more complex and contain more than a single level of nested objects: 168 169```nginx 170section "blah" "foo" { 171 key = value; 172} 173``` 174 175is presented as: 176 177```nginx 178section { 179 blah { 180 foo { 181 key = value; 182 } 183 } 184} 185``` 186 187### Convenient numbers and booleans 188 189* Numbers can have suffixes to specify standard multipliers: 190 + `[kKmMgG]` - standard 10 base multipliers (so `1k` is translated to 1000) 191 + `[kKmMgG]b` - 2 power multipliers (so `1kb` is translated to 1024) 192 + `[s|min|d|w|y]` - time multipliers, all time values are translated to float number of seconds, for example `10min` is translated to 600.0 and `10ms` is translated to 0.01 193* Hexadecimal integers can be used by `0x` prefix, for example `key = 0xff`. However, floating point values can use decimal base only. 194* Booleans can be specified as `true` or `yes` or `on` and `false` or `no` or `off`. 195* It is still possible to treat numbers and booleans as strings by enclosing them in double quotes. 196 197## General improvements 198 199### Commments 200 201UCL supports different style of comments: 202 203* single line: `#` 204* multiline: `/* ... */` 205 206Multiline comments may be nested: 207```c 208# Sample single line comment 209/* 210 some comment 211 /* nested comment */ 212 end of comment 213*/ 214``` 215 216### Macros support 217 218UCL supports external macros both multiline and single line ones: 219```nginx 220.macro "sometext"; 221.macro { 222 Some long text 223 .... 224}; 225``` 226There are two internal macros provided by UCL: 227 228* `include` - read a file `/path/to/file` or an url `http://example.com/file` and include it to the current place of 229UCL configuration; 230* `try\_include` - try to read a file or url and include it but do not create a fatal error if a file or url is not accessible; 231* `includes` - read a file or an url like the previous macro, but fetch and check the signature file (which is obtained 232by `.sig` suffix appending). 233 234Public keys which are used for the last command are specified by the concrete UCL user. 235 236### Variables support 237 238UCL supports variables in input. Variables are registered by a user of the UCL parser and can be presented in the following forms: 239 240* `${VARIABLE}` 241* `$VARIABLE` 242 243UCL currently does not support nested variables. To escape variables one could use double dollar signs: 244 245* `$${VARIABLE}` is converted to `${VARIABLE}` 246* `$$VARIABLE` is converted to `$VARIABLE` 247 248However, if no valid variables are found in a string, no expansion will be performed (and `$$` thus remains unchanged). This may be a subject 249to change in future libucl releases. 250 251### Multiline strings 252 253UCL can handle multiline strings as well as single line ones. It uses shell/perl like notation for such objects: 254``` 255key = <<EOD 256some text 257splitted to 258lines 259EOD 260``` 261 262In this example `key` will be interpreted as the following string: `some text\nsplitted to\nlines`. 263Here are some rules for this syntax: 264 265* Multiline terminator must start just after `<<` symbols and it must consist of capital letters only (e.g. `<<eof` or `<< EOF` won't work); 266* Terminator must end with a single newline character (and no spaces are allowed between terminator and newline character); 267* To finish multiline string you need to include a terminator string just after newline and followed by a newline (no spaces or other characters are allowed as well); 268* The initial and the final newlines are not inserted to the resulting string, but you can still specify newlines at the begin and at the end of a value, for example: 269 270``` 271key <<EOD 272 273some 274text 275 276EOD 277``` 278 279## Emitter 280 281Each UCL object can be serialized to one of the three supported formats: 282 283* `JSON` - canonic json notation (with spaces indented structure); 284* `Compacted JSON` - compact json notation (without spaces or newlines); 285* `Configuration` - nginx like notation; 286* `YAML` - yaml inlined notation. 287 288## Validation 289 290UCL allows validation of objects. It uses the same schema that is used for json: [json schema v4](http://json-schema.org). UCL supports the full set of json schema with the exception of remote references. This feature is unlikely useful for configuration objects. Of course, a schema definition can be in UCL format instead of JSON that simplifies schemas writing. Moreover, since UCL supports multiple values for keys in an object it is possible to specify generic integer constraints `maxValues` and `minValues` to define the limits of values count in a single key. UCL currently is not absolutely strict about validation schemas themselves, therefore UCL users should supply valid schemas (as it is defined in json-schema draft v4) to ensure that the input objects are validated properly. 291 292## Performance 293 294Are UCL parser and emitter fast enough? Well, there are some numbers. 295I got a 19Mb file that consist of ~700 thousands lines of json (obtained via 296http://www.json-generator.com/). Then I checked jansson library that performs json 297parsing and emitting and compared it with UCL. Here are results: 298 299``` 300jansson: parsed json in 1.3899 seconds 301jansson: emitted object in 0.2609 seconds 302 303ucl: parsed input in 0.6649 seconds 304ucl: emitted config in 0.2423 seconds 305ucl: emitted json in 0.2329 seconds 306ucl: emitted compact json in 0.1811 seconds 307ucl: emitted yaml in 0.2489 seconds 308``` 309 310So far, UCL seems to be significantly faster than jansson on parsing and slightly faster on emitting. Moreover, 311UCL compiled with optimizations (-O3) performs significantly faster: 312``` 313ucl: parsed input in 0.3002 seconds 314ucl: emitted config in 0.1174 seconds 315ucl: emitted json in 0.1174 seconds 316ucl: emitted compact json in 0.0991 seconds 317ucl: emitted yaml in 0.1354 seconds 318``` 319 320You can do your own benchmarks by running `make test` in libucl top directory. 321 322## Conclusion 323 324UCL has clear design that should be very convenient for reading and writing. At the same time it is compatible with 325JSON language and therefore can be used as a simple JSON parser. Macroes logic provides an ability to extend configuration 326language (for example by including some lua code) and comments allows to disable or enable the parts of a configuration 327quickly. 328