Parser

Supported Conditional Functions

Comprehensive reference for parser conditional functions

This document provides a comprehensive reference for all functions available in the #conditional block of parser configurations.

Timestamp functions

format_timestamp

infallible pure

Formats value into a string representation of the timestamp.

Function spec

format_timestamp(value: <timestamp>, format: <string>, [timezone: <string>])
  :: <string>

Argument	Type	Description	Required?
value	timestamp	The timestamp to format as text.	yes
format	string	The format string as described by the Chrono library.	yes
timezone	string	The timezone to use when formatting the timestamp.	no

Examples

Format a timestamp (ISO8601/RFC 3339)

Source

format_timestamp!(t'2020-10-21T16:00:00Z', format: "%+")

Return

2020-10-21T16:00:00+00:00

now

infallible pure

Returns the current timestamp in the UTC timezone with nanosecond precision.

Function spec

now()
  :: <timestamp>

Coerce functions

to_float

fallible pure

Coerces the value into a float.

Function spec

to_float(value: <integer | float | boolean | string | timestamp>)
  :: <float>, <error>

Argument	Type	Description	Default	Required?
value	integer, float, boolean, string, timestamp	The value to convert to a float. Must be convertible to a float, otherwise an error is raised.		yes

Errors

The to_float function is fallible, which means that error handling is required for these errors:

⚠️ value is not a supported float representation.

Examples

Coerce to a float

Source

to_float!("3.145")

Return

3.145

Coerce to a float (timestamp)

Source

to_float(t'2020-12-30T22:20:53.824727Z')

Return

1609366853.824727

to_int

fallible pure

Coerces the value into an integer.

Function spec

to_int(value: <integer | float | boolean | string | timestamp | null>)
  :: <integer>, <error>

Argument	Type	Description	Default	Required?
value	integer, float, boolean, string, timestamp, null	The value to convert to an integer.		yes

Errors

The to_int function is fallible, which means that error handling is required for these errors:

⚠️ value is a string but the text is not an integer, or value is not a string, int, or timestamp.

Examples

Coerce to an int (string)

Source

to_int!("2")

Return

Coerce to an int (timestamp)

Source

to_int(t'2020-12-30T22:20:53.824727Z')

Return

1609366853

to_regex

fallible pure

Coerces the value into a regex.

Function spec

to_regex(value: <string>)
  :: <regex>, <error>

Argument	Type	Description	Default	Required?
value	string	The value to convert to a regex.		yes

to_string

fallible pure

Coerces the value into a string.

Function spec

to_string(value: <integer | float | boolean | string | timestamp | null>)
  :: <string>, <error>

Argument	Type	Description	Default	Required?
value	integer, float, boolean, string, timestamp, null	The value to convert to a string.		yes

Errors

The to_string function is fallible, which means that error handling is required for these errors:

⚠️ value is not an integer, float, boolean, string, timestamp, or null.

Examples

Coerce to a string (Boolean)

Source

to_string(true)

Return

true

Coerce to a string (int)

Source

to_string(52)

Return

Coerce to a string (float)

Source

to_string(52.2)

Return

52.2

String Manipulation functions

Argument	Type	Description	Default	Required?
value	array, object, string	The array or object.		yes

contains

infallible pure

Determines whether the value string contains the specified substring.

Function spec

contains(value: <string>, substring: <string>, [case_sensitive: <boolean>])
  :: <boolean>

Argument	Type	Description	Default	Required?
value	string	The text to search.		yes
substring	string	The substring to search for in `value`.		yes
case_sensitive	boolean	Whether the match should be case sensitive.	true	no

Examples

String contains (case sensitive)

Source

contains("The Needle In The Haystack", "Needle")

Return

true

String contains (case insensitive)

Source

contains("The Needle In The Haystack", "needle", case_sensitive: false)

Return

true

downcase

infallible pure

Downcases the value string, where downcase is defined according to the Unicode Derived Core Property Lowercase.

Function spec

downcase(value: <string>)
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to convert to lowercase.		yes

Examples

Downcase a string

Source

downcase("Hello, World!")

Return

hello, world!

ends_with

infallible pure

Determines whether the value string ends with the specified substring.

Function spec

ends_with(value: <string>, substring: <string>, [case_sensitive: <boolean>])
  :: <boolean>

Argument	Type	Description	Default	Required?
value	string	The string to search.		yes
substring	string	The substring with which `value` must end.		yes
case_sensitive	boolean	Whether the match should be case sensitive.	true	no

Examples

String ends with (case sensitive)

Source

ends_with("The Needle In The Haystack", "The Haystack")

Return

true

String ends with (case insensitive)

Source

ends_with("The Needle In The Haystack", "the haystack", case_sensitive: false)

Return

true

replace

infallible pure

Replaces all matching instances of pattern in value. The pattern argument accepts regular expression capture groups.

Function spec

replace(value: <string>, pattern: <regex | string>, with: <string>, [count: <integer>])
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The original string.		yes
pattern	regex, string	Replace all matches of this pattern.		yes
with	string	The string that the matches are replaced with.		yes
count	integer	The maximum number of replacements to perform.	-1	no

Examples

Replace literal text

Source

replace("Apples and Bananas", "and", "not")

Return

Apples not Bananas

Replace using regular expression

Source

replace("Apples and Bananas", r'(?i)bananas', "Pineapples")

Return

Apples and Pineapples

split

infallible pure

Splits the value string using pattern.

Function spec

split(value: <string>, pattern: <string | regex>, [limit: <integer>])
  :: <array>

Argument	Type	Description	Required?
value	string	The string to split.	yes
pattern	string, regex	The string is split whenever this pattern is matched.	yes
limit	integer	The maximum number of substrings to return.	no

Examples

Split a string (no limit)

Source

split("apples and pears and bananas", " and ")

Return

["apples", "pears", "bananas"]

starts_with

infallible pure

Determines whether value starts with substring.

Function spec

starts_with(value: <string>, substring: <string>, [case_sensitive: <boolean>])
  :: <boolean>

Argument	Type	Description	Default	Required?
value	string	The string to search.		yes
substring	string	The substring that the `value` must start with.		yes
case_sensitive	boolean	Whether the match should be case sensitive.	true	no

Examples

String starts with (case sensitive)

Source

starts_with("The Needle In The Haystack", "The Needle")

Return

true

String starts with (case insensitive)

Source

starts_with("The Needle In The Haystack", "the needle", case_sensitive: false)

Return

true

upcase

infallible pure

Upcases value, where upcase is defined according to the Unicode Derived Core Property Uppercase.

Function spec

upcase(value: <string>)
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to convert to uppercase.		yes

Examples

Upcase a string

Source

upcase("Hello, World!")

Return

HELLO, WORLD!

Cryptography functions

md5

infallible pure

Calculates an md5 hash of the value.

Function spec

md5(value: <string>)
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to calculate the hash for.		yes

Examples

Create md5 hash

Source

md5("foo")

Return

acbd18db4cc2f85cedef654fccc4a4d8

sha1

infallible pure

Calculates a SHA-1 hash of the value.

Function spec

sha1(value: <string>)
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to calculate the hash for.		yes

Examples

Calculate sha1 hash

Source

sha1("foo")

Return

0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33

sha2

infallible pure

Calculates a SHA-2 hash of the value.

Function spec

sha2(value: <string>, [variant: <string>])
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to calculate the hash for.		yes
variant	string	The variant of the algorithm to use.	SHA-512/256	no

Examples

Calculate sha2 hash

Source

sha2("foo", variant: "SHA-512/224")

Return

d68f258d37d670cfc1ec1001a0394784233f88f056994f9a7e5e99be

decode_base64

fallible pure

Decodes the value (a Base64 string) into its original string.

Function spec

decode_base64(value: <string>, [charset: <string>])
  :: <string>, <error>

Argument	Type	Description	Default	Required?
value	string	The Base64 data to decode.		yes
charset	string	The character set to use when decoding the data.	standard	no

encode_base64

infallible pure

Encodes the value to Base64.

Function spec

encode_base64(value: <string>, [padding: <boolean>, charset: <string>])
  :: <string>

Argument	Type	Description	Default	Required?
value	string	The string to encode.		yes
padding	boolean	Whether the Base64 output is padded.	true	no
charset	string	The character set to use when encoding the data.	standard	no

Examples

Encode to Base64 (default)

Source

encode_base64("please encode me")

Return

cGxlYXNlIGVuY29kZSBtZQ==

Encode to Base64 (without padding)

Source

encode_base64("please encode me, no padding though", padding: false)

Return

cGxlYXNlIGVuY29kZSBtZSwgbm8gcGFkZGluZyB0aG91Z2g

Encode to Base64 (URL safe)

Source

encode_base64("please encode me, but safe for URLs", charset: "url_safe")

Return

cGxlYXNlIGVuY29kZSBtZSwgYnV0IHNhZmUgZm9yIFVSTHM=

Random functions

uuid_v4

infallible pure

Generates a random UUIDv4 string.

Function spec

uuid_v4()
  :: <string>

Examples

Create a UUIDv4

Source

uuid_v4()

Return

1d262f4f-199b-458d-879f-05fd0a5f0683

Conditional Logic

if(condition, {block}, else {block}): Executes block if condition is true; otherwise, executes the else block.
and(condition1, condition2): Evaluates to true if both condition1 and condition2 are true.
else: Used within if to define fallback logic.

Parse functions

parse_apache_log

fallible pure

Parses Apache access and error log lines. Lines can be in common, combined, or the default error format.

Function spec

parse_apache_log(value: <string>, format: <string>, [timestamp_format: <string>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
format	string	The format to use for parsing the log.		yes
timestamp_format	string	The date/time format to use for encoding the timestamp.	%d/%b/%Y:%T %z	no

Notices

Missing information in the log message may be indicated by -, and these fields are omitted in the result.

Errors

The parse_apache_log function is fallible, requiring error handling if value does not match the format, timestamp_format is invalid, or the timestamp fails to parse.

Examples

Parse using Apache log format (common)

Source

parse_apache_log!("127.0.0.1 bob frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326", format: "common")

Return

{
  "host": "127.0.0.1",
  "identity": "bob",
  "message": "GET /apache_pb.gif HTTP/1.0",
  "method": "GET",
  "path": "/apache_pb.gif",
  "protocol": "HTTP/1.0",
  "size": 2326,
  "status": 200,
  "timestamp": "2000-10-10T20:55:36Z",
  "user": "frank"
}

parse_aws_alb_log

fallible pure

Parses value in the Elastic Load Balancer Access format.

Function spec

parse_aws_alb_log(value: <string>, [strict_mode: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	Access log of the Application Load Balancer.		yes
strict_mode	boolean	When false, the parser ignores newly added or trailing fields instead of failing. Defaults to true.	true	no

Errors

The parse_aws_alb_log function is fallible, requiring error handling if value is not a properly formatted AWS ALB log.

Examples

Parse AWS ALB log

Source

parse_aws_alb_log!("http 2018-11-30T22:23:00.186641Z app/my-loadbalancer/50dc6c495c0c9188 192.168.131.39:2817 - 0.000 0.001 0.000 200 200 34 366 \"GET http://www.example.com:80/ HTTP/1.1\" \"curl/7.46.0\" - - arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067 \"Root=1-58337364-23a8c76965a2ef7629b185e3\" \"-\" \"-\" 0 2018-11-30T22:22:48.364000Z \"forward\" \"-\" \"-\" \"-\" \"-\" \"-\" \"-\"")

Return

{
  "actions_executed": "forward",
  "chosen_cert_arn": null,
  "classification": null,
  "classification_reason": null,
  "client_host": "192.168.131.39:2817",
  "domain_name": null,
  "elb": "app/my-loadbalancer/50dc6c495c0c9188",
  "elb_status_code": "200",
  "error_reason": null,
  "matched_rule_priority": "0",
  "received_bytes": 34,
  "redirect_url": null,
  "request_creation_time": "2018-11-30T22:22:48.364000Z",
  "request_method": "GET",
  "request_processing_time": 0,
  "request_protocol": "HTTP/1.1",
  "request_url": "http://www.example.com:80/",
  "response_processing_time": 0,
  "sent_bytes": 366,
  "ssl_cipher": null,
  "ssl_protocol": null,
  "target_group_arn": "arn:aws:elasticloadbalancing:us-east-2:123456789012:targetgroup/my-targets/73e2d6bc24d8a067",
  "target_host": null,
  "target_port_list": [],
  "target_processing_time": 0.001,
  "target_status_code": "200",
  "target_status_code_list": [],
  "timestamp": "2018-11-30T22:23:00.186641Z",
  "trace_id": "Root=1-58337364-23a8c76965a2ef7629b185e3",
  "traceability_id": null,
  "type": "http",
  "user_agent": "curl/7.46.0"
}

parse_aws_cloudwatch_log_subscription_message

fallible pure

Parses AWS CloudWatch Logs events (configured through AWS Cloudwatch subscriptions) from the aws_kinesis_firehose source.

Function spec

parse_aws_cloudwatch_log_subscription_message(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string representation of the message to parse.		yes

Errors

The parse_aws_cloudwatch_log_subscription_message function is fallible, requiring error handling if value is not a properly formatted AWS CloudWatch Log subscription message.

Examples

Parse AWS Cloudwatch Log subscription message

Source

parse_aws_cloudwatch_log_subscription_message!(.message)

Return

{
  "log_events": [
    {
      "id": "35683658089614582423604394983260738922885519999578275840",
      "message": "{\"bytes\":26780,\"datetime\":\"14/Sep/2020:11:45:41 -0400\",\"host\":\"157.130.216.193\",\"method\":\"PUT\",\"protocol\":\"HTTP/1.0\",\"referer\":\"https://www.principalcross-platform.io/markets/ubiquitous\",\"request\":\"/expedite/convergence\",\"source_type\":\"stdin\",\"status\":301,\"user-identifier\":\"-\"}",
      "timestamp": "2020-09-14T19:09:29.039Z"
    }
  ],
  "log_group": "test",
  "log_stream": "test",
  "message_type": "DATA_MESSAGE",
  "owner": "111111111111",
  "subscription_filters": ["Destination"]
}

parse_aws_vpc_flow_log

fallible pure

Parses value in the VPC Flow Logs format.

Function spec

parse_aws_vpc_flow_log(value: <string>, [format: <string>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	VPC Flow Log.		yes
format	string	VPC Flow Log format.		no

Errors

The parse_aws_vpc_flow_log function is fallible, requiring error handling if value is not a properly formatted AWS VPC Flow log.

Examples

Parse AWS VPC Flow log (default format)

Source

parse_aws_vpc_flow_log!("2 123456789010 eni-1235b8ca123456789 - - - - - - - 1431280876 1431280934 - NODATA")

Return

{
  "account_id": "123456789010",
  "action": null,
  "bytes": null,
  "dstaddr": null,
  "dstport": null,
  "end": 1431280934,
  "interface_id": "eni-1235b8ca123456789",
  "log_status": "NODATA",
  "packets": null,
  "protocol": null,
  "srcaddr": null,
  "srcport": null,
  "start": 1431280876,
  "version": 2
}

Parse AWS VPC Flow log (custom format)

Source

parse_aws_vpc_flow_log!("- eni-1235b8ca123456789 10.0.1.5 10.0.0.220 10.0.1.5 203.0.113.5", "instance_id interface_id srcaddr dstaddr pkt_srcaddr pkt_dstaddr")

Return

{
  "dstaddr": "10.0.0.220",
  "instance_id": null,
  "interface_id": "eni-1235b8ca123456789",
  "pkt_dstaddr": "203.0.113.5",
  "pkt_srcaddr": "10.0.1.5",
  "srcaddr": "10.0.1.5"
}

Argument	Type	Description	Default	Required?
value	string	The string of the duration with either binary or SI unit.		yes
unit	string	The output units for the byte.		yes
base	string	The base for the byte, either 2 or 10.	2	no

parse_cbor

fallible pure

Parses the value as CBOR.

Function spec

parse_cbor(value: <string>)
  :: <boolean | integer | float | string | object | array | null>, <error>

Argument	Type	Description	Default	Required?
value	string	The CBOR payload to parse.		yes

Errors

The parse_cbor function is fallible, requiring error handling if value is not a valid CBOR-formatted payload.

Examples

Parse CBOR

Source

parse_cbor!(decode_base64!("oWVmaWVsZGV2YWx1ZQ=="))

Return

{"field": "value"}

parse_cef

fallible pure

Parses the value in CEF (Common Event Format) format. Ignores everything up to the CEF header.

Function spec

parse_cef(value: <string>, [translate_custom_fields: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
translate_custom_fields	boolean	Toggles translation of custom field pairs to key:value.		no

Errors

The parse_cef function is fallible, requiring error handling if value is not a properly formatted CEF string.

Examples

Parse output generated by PTA

Source

parse_cef!("CEF:0|CyberArk|PTA|12.6|1|Suspected credentials theft|8|suser=mike2@prod1.domain.com shost=prod1.domain.com src=1.1.1.1 duser=andy@dev1.domain.com dhost=dev1.domain.com dst=2.2.2.2 cs1Label=ExtraData cs1=None cs2Label=EventID cs2=52b06812ec3500ed864c461e deviceCustomDate1Label=detectionDate deviceCustomDate1=1388577900000 cs3Label=PTAlink cs3=https://1.1.1.1/incidents/52b06812ec3500ed864c461e cs4Label=ExternalLink cs4=None")

Return

{
  "cefVersion": "0",
  "cs1": "None",
  "cs1Label": "ExtraData",
  "cs2": "52b06812ec3500ed864c461e",
  "cs2Label": "EventID",
  "cs3": "https://1.1.1.1/incidents/52b06812ec3500ed864c461e",
  "cs3Label": "PTAlink",
  "cs4": "None",
  "cs4Label": "ExternalLink",
  "deviceCustomDate1": "1388577900000",
  "deviceCustomDate1Label": "detectionDate",
  "deviceEventClassId": "1",
  "deviceProduct": "PTA",
  "deviceVendor": "CyberArk",
  "deviceVersion": "12.6",
  "dhost": "dev1.domain.com",
  "dst": "2.2.2.2",
  "duser": "andy@dev1.domain.com",
  "name": "Suspected credentials theft",
  "severity": "8",
  "shost": "prod1.domain.com",
  "src": "1.1.1.1",
  "suser": "mike2@prod1.domain.com"
}

Translate custom fields

Source

parse_cef!("CEF:0|Dev|firewall|2.2|1|Connection denied|5|c6a1=2345:0425:2CA1:0000:0000:0567:5673:23b5 c6a1Label=Device IPv6 Address", translate_custom_fields: true)

Return

{
  "Device IPv6 Address": "2345:0425:2CA1:0000:0000:0567:5673:23b5",
  "cefVersion": "0",
  "deviceEventClassId": "1",
  "deviceProduct": "firewall",
  "deviceVendor": "Dev",
  "deviceVersion": "2.2",
  "name": "Connection denied",
  "severity": "5"
}

parse_common_log

fallible pure

Parses the value using the Common Log Format (CLF).

Function spec

parse_common_log(value: <string>, [timestamp_format: <string>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
timestamp_format	string	The date/time format to use for encoding the timestamp.	%d/%b/%Y:%T %z	no

Notices

Missing information in the log message may be indicated by -, and these fields are omitted in the result.

Errors

The parse_common_log function is fallible, requiring error handling if value does not match CLF, timestamp_format is invalid, or the timestamp fails to parse.

Examples

Parse using Common Log Format (with default timestamp format)

Source

parse_common_log!("127.0.0.1 bob frank [10/Oct/2000:13:55:36 -0700] \"GET /apache_pb.gif HTTP/1.0\" 200 2326")

Return

{
  "host": "127.0.0.1",
  "identity": "bob",
  "message": "GET /apache_pb.gif HTTP/1.0",
  "method": "GET",
  "path": "/apache_pb.gif",
  "protocol": "HTTP/1.0",
  "size": 2326,
  "status": 200,
  "timestamp": "2000-10-10T20:55:36Z",
  "user": "frank"
}

parse_csv

fallible pure

Parses a single CSV formatted row. Only the first row is parsed in case of multiline input value.

Function spec

parse_csv(value: <string>, [delimiter: <string>])
  :: <array>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
delimiter	string	The field delimiter to use when parsing.	,	no

Errors

The parse_csv function is fallible, requiring error handling if the delimiter is not a single-byte UTF-8 character, or value is not a valid CSV string.

Examples

Parse a single CSV formatted row

Source

parse_csv!("foo,bar,\"foo\"\", bar\"")

Return

["foo", "bar", "foo\", bar"]

parse_dnstap

fallible pure

Parses the value as base64 encoded DNSTAP data.

Function spec

parse_dnstap(value: <string>, [lowercase_hostnames: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The base64 encoded representation of the DNSTAP data to parse.		yes
lowercase_hostnames	boolean	Whether to turn all hostnames found in resulting data lowercase.		no

Errors

The parse_dnstap function is fallible, requiring error handling if value is not a valid base64 encoded string, or dnstap parsing failed.

Examples

Parse dnstap query message

Source

parse_dnstap!("ChVqYW1lcy1WaXJ0dWFsLU1hY2hpbmUSC0JJTkQgOS4xNi4zGgBy5wEIAxACGAEiEAAAAAAAAAAAAAAAAAAAAAAqECABBQJwlAAAAAAAAAAAADAw8+0CODVA7+zq9wVNMU3WNlI2kwIAAAABAAAAAAABCWZhY2Vib29rMQNjb20AAAEAAQAAKQIAAACAAAAMAAoACOxjCAG9zVgzWgUDY29tAGAAbQAAAAByZLM4AAAAAQAAAAAAAQJoNQdleGFtcGxlA2NvbQAABgABAAApBNABAUAAADkADwA1AAlubyBTRVAgbWF0Y2hpbmcgdGhlIERTIGZvdW5kIGZvciBkbnNzZWMtZmFpbGVkLm9yZy54AQ==")

Return

{
  "dataType": "Message",
  "dataTypeId": 1,
  "extraInfo": "",
  "messageType": "ResolverQuery",
  "messageTypeId": 3,
  "queryZone": "com.",
  "requestData": {
    "fullRcode": 0,
    "header": {
      "aa": false,
      "ad": false,
      "anCount": 0,
      "arCount": 1,
      "cd": false,
      "id": 37634,
      "nsCount": 0,
      "opcode": 0,
      "qdCount": 1,
      "qr": 0,
      "ra": false,
      "rcode": 0,
      "rd": false,
      "tc": false
    },
    "opt": {
      "do": true,
      "ednsVersion": 0,
      "extendedRcode": 0,
      "options": [{"optCode": 10, "optName": "Cookie", "optValue": "7GMIAb3NWDM="}],
      "udpPayloadSize": 512
    },
    "question": [{"class": "IN", "domainName": "facebook1.com.", "questionType": "A", "questionTypeId": 1}],
    "rcodeName": "NoError"
  },
  "responseAddress": "2001:502:7094::30",
  "responsePort": 53,
  "serverId": "james-Virtual-Machine",
  "serverVersion": "BIND 9.16.3",
  "socketFamily": "INET6",
  "socketProtocol": "UDP",
  "sourceAddress": "::",
  "sourcePort": 46835,
  "time": 1593489007920014000,
  "timePrecision": "ns",
  "timestamp": "2020-06-30T03:50:07.920014129Z"
}

Argument	Type	Description	Default	Required?
value	string	The string of the duration.		yes
unit	string	The output units for the duration.		yes

parse_etld

fallible pure

Parses the eTLD from value representing domain name.

Function spec

parse_etld(value: <string>, [plus_parts: <integer>, psl: <string>])
  :: <object>, <error>

Argument	Type	Description	Required?
value	string	The domain string.	yes
plus_parts	integer	Can be provided to get additional parts of the domain name.	no
psl	string	Can be provided to use a different public suffix list.	no

Errors

The parse_etld function is fallible, requiring error handling if unable to determine eTLD for value.

Examples

Parse eTLD

Source

parse_etld!("sub.sussex.ac.uk")

Return

{"etld": "ac.uk", "etld_plus": "ac.uk", "known_suffix": true}

Parse eTLD+1

Source

parse_etld!("sub.sussex.ac.uk", plus_parts: 1)

Return

{"etld": "ac.uk", "etld_plus": "sussex.ac.uk", "known_suffix": true}

parse_glog

fallible pure

Parses the value using the glog (Google Logging Library) format.

Function spec

parse_glog(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes

Errors

The parse_glog function is fallible, requiring error handling if value does not match the glog format.

Examples

Parse using glog

Source

parse_glog!("I20210131 14:48:54.411655 15520 main.c++:9] Hello world!")

Return

{
  "file": "main.c++",
  "id": 15520,
  "level": "info",
  "line": 9,
  "message": "Hello world!",
  "timestamp": "2021-01-31T14:48:54.411655Z"
}

parse_grok

fallible pure

Parses the value using the grok format.

Function spec

parse_grok(value: <string>, pattern: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
pattern	string	The Grok pattern.		yes

Notices

It is recommended to use community-maintained Grok patterns.

Errors

The parse_grok function is fallible, requiring error handling if value fails to parse using the provided pattern.

Examples

Parse using Grok

Source

parse_grok!("2020-10-02T23:22:12.223222Z info Hello world", "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:message}")

Return

{
  "level": "info",
  "message": "Hello world",
  "timestamp": "2020-10-02T23:22:12.223222Z"
}

parse_groks

fallible pure

Parses the value using multiple grok patterns. Patterns are tried in order until the first match.

Function spec

parse_groks(value: <string>, patterns: <array>, [aliases: <object>, alias_sources: <string>])
  :: <object>, <error>

Argument	Type	Description	Required?
value	string	The string to parse.	yes
patterns	array	The Grok patterns, which are tried in order until the first match.	yes
aliases	object	The shared set of grok aliases that can be referenced in the patterns.	no
alias_sources	string	Path to the file containing aliases in a JSON format.	no

Notices

It is recommended to use community-maintained Grok patterns.

Errors

The parse_groks function is fallible, requiring error handling if value fails to parse, or parameters (patterns, aliases, alias_sources) are invalid.

Examples

Parse using multiple Grok patterns

Source

parse_groks!("2020-10-02T23:22:12.223222Z info Hello world",
  patterns: [
    "%{common_prefix} %{_status} %{_message}",
    "%{common_prefix} %{_message}",
  ],
  aliases: {
    "common_prefix": "%{_timestamp} %{_loglevel}",
    "_timestamp": "%{TIMESTAMP_ISO8601:timestamp}",
    "_loglevel": "%{LOGLEVEL:level}",
    "_status": "%{POSINT:status}",
    "_message": "%{GREEDYDATA:message}"
  }
)

Return

{
  "level": "info",
  "message": "Hello world",
  "timestamp": "2020-10-02T23:22:12.223222Z"
}

parse_influxdb

fallible pure

Parses the value as an InfluxDB line protocol string, producing a list of Vector-compatible metrics.

Function spec

parse_influxdb(value: <string>)
  :: <array>, <error>

Argument	Type	Description	Default	Required?
value	string	The string representation of the InfluxDB line protocol to parse.		yes

Notices

This function returns a log event shaped like a Vector-compatible metric, not a metric event itself. Only the gauge metric type is produced. string is the only type not supported as a field value.

Errors

The parse_influxdb function is fallible, requiring error handling if value is not a valid InfluxDB line protocol string, contains a string field value, or contains a NaN field value.

Examples

Parse InfluxDB line protocol

Source

parse_influxdb!("cpu,host=A,region=us-west usage_system=64i,usage_user=10u,temperature=50.5,on=true,sleep=false 1590488773254420000")

Return

[
  {"gauge":{"value":64},"kind":"absolute","name":"cpu_usage_system","tags":{"host":"A","region":"us-west"},"timestamp":"2020-05-26T10:26:13.254420Z"},
  {"gauge":{"value":10},"kind":"absolute","name":"cpu_usage_user","tags":{"host":"A","region":"us-west"},"timestamp":"2020-05-26T10:26:13.254420Z"},
  {"gauge":{"value":50.5},"kind":"absolute","name":"cpu_temperature","tags":{"host":"A","region":"us-west"},"timestamp":"2020-05-26T10:26:13.254420Z"},
  {"gauge":{"value":1},"kind":"absolute","name":"cpu_on","tags":{"host":"A","region":"us-west"},"timestamp":"2020-05-26T10:26:13.254420Z"},
  {"gauge":{"value":0},"kind":"absolute","name":"cpu_sleep","tags":{"host":"A","region":"us-west"},"timestamp":"2020-05-26T10:26:13.254420Z"}
]

parse_json

fallible pure

Parses the value as JSON.

Function spec

parse_json(value: <string>, [max_depth: <integer>, lossy: <boolean>])
  :: <boolean | integer | float | string | object | array | null>, <error>

Argument	Type	Description	Default	Required?
value	string	The string representation of the JSON to parse.		yes
max_depth	integer	Number of layers to parse for nested JSON-formatted documents (range 1 to 128).		no
lossy	boolean	Whether to parse the JSON in a lossy manner, replacing invalid UTF-8 characters.	true	no

Errors

The parse_json function is fallible, requiring error handling if value is not a valid JSON-formatted payload.

Examples

Parse JSON

Source

parse_json!("{\"key\": \"val\"}")

Return

{"key": "val"}

Parse JSON with max_depth

Source

parse_json!("{\"top_level\":{\"key\": \"val\"}}", max_depth: 1)

Return

{"top_level": "{\"key\": \"val\"}"}

parse_key_value

fallible pure

Parses the value in key-value format, also known as logfmt.

Function spec

parse_key_value(value: <string>, [key_value_delimiter: <string>, field_delimiter: <string>, whitespace: <string>, accept_standalone_key: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
key_value_delimiter	string	The string that separates the key from the value.	=	no
field_delimiter	string	The string that separates each key-value pair.	(space)	no
accept_standalone_key	boolean	Whether a standalone key should be accepted (associated with boolean true).	true	no

Notices

All values are returned as strings or as an array of strings for duplicate keys.

Errors

The parse_key_value function is fallible, requiring error handling if value is not a properly formatted key-value string.

Examples

Parse logfmt log

Source

parse_key_value!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager")

Return

{
  "@timestamp": "Sun Jan 10 16:47:39 EST 2021",
  "id": "ConsumerFetcherManager-1382721708341",
  "level": "info",
  "module": "kafka.consumer.ConsumerFetcherManager",
  "msg": "Stopping all fetchers",
  "tag#production": "stopping_fetchers"
}

Parse comma delimited log with standalone keys

Source

parse_key_value!("env:prod,service:backend,region:eu-east1,beta", field_delimiter: ",", key_value_delimiter: ":")

Return

{
  "beta": true,
  "env": "prod",
  "region": "eu-east1",
  "service": "backend"
}

parse_klog

fallible pure

Parses the value using the klog format used by Kubernetes components.

Function spec

parse_klog(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes

Notices

This function resolves the year for messages: if the current month is January and the provided month is December, it sets the year to the previous year, otherwise, it uses the current year.

Errors

The parse_klog function is fallible, requiring error handling if value does not match the klog format.

Examples

Parse using klog

Source

parse_klog!("I0505 17:59:40.692994 28133 klog.go:70] hello from klog")

Return

{
  "file": "klog.go",
  "id": 28133,
  "level": "info",
  "line": 70,
  "message": "hello from klog",
  "timestamp": "2025-05-05T17:59:40.692994Z"
}

parse_linux_authorization

fallible pure

Parses Linux authorization logs usually found under either /var/log/auth.log or /var/log/secure according to Syslog format.

Function spec

parse_linux_authorization(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The text containing the message to parse.		yes

Notices

The function resolves the year for messages that don't include it based on the current month.

parse_logfmt

fallible pure

Parses the value in logfmt. The function accepts standalone keys and assigns them a Boolean value of true.

Function spec

parse_logfmt(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes

Errors

The parse_logfmt function is fallible, requiring error handling if value is not a properly formatted key-value string.

Examples

Parse logfmt log

Source

parse_logfmt!("@timestamp=\"Sun Jan 10 16:47:39 EST 2021\" level=info msg=\"Stopping all fetchers\" tag#production=stopping_fetchers id=ConsumerFetcherManager-1382721708341 module=kafka.consumer.ConsumerFetcherManager")

Return

{
  "@timestamp": "Sun Jan 10 16:47:39 EST 2021",
  "id": "ConsumerFetcherManager-1382721708341",
  "level": "info",
  "module": "kafka.consumer.ConsumerFetcherManager",
  "msg": "Stopping all fetchers",
  "tag#production": "stopping_fetchers"
}

parse_nginx_log

fallible pure

Parses Nginx access and error log lines. Lines can be in combined, ingress_upstreaminfo, main or error format.

Function spec

parse_nginx_log(value: <string>, format: <string>, [timestamp_format: <string>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
format	string	The format to use for parsing the log.		yes
timestamp_format	string	The date/time format to use for encoding the timestamp.	%d/%b/%Y:%T %z	no

Notices

Missing information in the log message may be indicated by -, and these fields are omitted in the result.

Errors

The parse_nginx_log function is fallible, requiring error handling if value does not match the specified format, timestamp_format is invalid, or the timestamp fails to parse.

Examples

Parse via Nginx log format (combined)

Source

parse_nginx_log!(s'172.17.0.1 - alice [01/Apr/2021:12:02:31 +0000] "POST /not-found HTTP/1.1" 404 153 "http://localhost/somewhere" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36" "2.75"', "combined")

Return

{
  "agent": "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36",
  "client": "172.17.0.1",
  "compression": "2.75",
  "referer": "http://localhost/somewhere",
  "request": "POST /not-found HTTP/1.1",
  "size": 153,
  "status": 404,
  "timestamp": "2021-04-01T12:02:31Z",
  "user": "alice"
}

parse_proto

fallible pure

Parses the value as a protocol buffer payload.

Function spec

parse_proto(value: <string>, desc_file: <string>, message_type: <string>)
  :: <object>, <error>

Argument	Type	Description	Required?
value	string	The protocol buffer payload to parse.	yes
desc_file	string	The path to the protobuf descriptor set file. Must be a literal string.	yes
message_type	string	The name of the message type to use for serializing. Must be a literal string.	yes

Notices

Only proto messages are parsed and returned.

The parse_proto function is fallible, requiring error handling if value is not a valid proto payload, desc_file file does not exist, or message_type message type does not exist in the descriptor file.

Examples

Parse proto

Source

parse_proto!(decode_base64!("Cgdzb21lb25lIggKBjEyMzQ1Ng=="), "resources/protobuf_descriptor_set.desc", "test_protobuf.Person")

Return

{
  "name": "someone",
  "phones": [{"number": "123456"}]
}

parse_query_string

infallible pure

Parses the value as a query string.

Function spec

parse_query_string(value: <string>)
  :: <object>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes

Notices

All values are returned as strings. Empty keys and values are allowed.

Examples

Parse query string

Source

parse_query_string("foo=%2B1&bar=2&bar=3&xyz")

Return

{"bar": ["2", "3"], "foo": "+1", "xyz": ""}

parse_regex

fallible pure

Parses the value using the provided Regex pattern. This function returns only the first match.

Function spec

parse_regex(value: <string>, pattern: <regex>, [numeric_groups: <regex>])
  :: <object>, <error>

Argument	Type	Description	Required?
value	string	The string to search.	yes
pattern	regex	The regular expression pattern to search against.	yes
numeric_groups	regex	If true, the index of each group in the regular expression is also captured.	no

Notices

All values are returned as strings.

Errors

The parse_regex function is fallible, requiring error handling if value fails to parse using the provided pattern.

Examples

Parse using Regex (with capture groups)

Source

parse_regex!("first group and second group.", r'(?P<number>.*?) group')

Return

{"number": "first"}

parse_regex_all

fallible pure

Parses the value using the provided Regex pattern. This function returns all matches, not just the first.

Function spec

parse_regex_all(value: <string>, pattern: <regex>, [numeric_groups: <regex>])
  :: <array>, <error>

Argument	Type	Description	Required?
value	string	The string to search.	yes
pattern	regex	The regular expression pattern to search against.	yes
numeric_groups	regex	If true, the index of each group in the regular expression is also captured.	no

parse_ruby_hash

fallible pure

Parses the value as a ruby hash.

Function spec

parse_ruby_hash(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string representation of the ruby hash to parse.		yes

Notices

Only ruby types are returned.

Errors

The parse_ruby_hash function is fallible, requiring error handling if value is not a valid ruby hash formatted payload.

Examples

Parse ruby hash

Source

parse_ruby_hash!(s'{ "test" => "value", "testNum" => 0.2, "testObj" => { "testBool" => true, "testNull" => nil } }')

Return

{
  "test": "value",
  "testNum": 0.2,
  "testObj": {"testBool": true, "testNull": null}
}

parse_syslog

fallible pure

Parses the value in Syslog format.

Function spec

parse_syslog(value: <string>)
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The text containing the Syslog message to parse.		yes

Examples

Parse Syslog log (5424)

Source

parse_syslog!(s'<13>1 2020-03-13T20:45:38.119Z dynamicwireless.name non 2426 ID931 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"] Try to override the THX port, maybe it will reboot the neural interface!')

Return

{
  "appname": "non",
  "exampleSDID@32473": {"eventID": "1011", "eventSource": "Application", "iut": "3"},
  "facility": "user",
  "hostname": "dynamicwireless.name",
  "message": "Try to override the THX port, maybe it will reboot the neural interface!",
  "msgid": "ID931",
  "procid": 2426,
  "severity": "notice",
  "timestamp": "2020-03-13T20:45:38.119Z",
  "version": 1
}

parse_timestamp

fallible pure

Parses the value in strptime format.

Function spec

parse_timestamp(value: <string>, format: <string>, [timezone: <string>])
  :: <timestamp>, <error>

Argument	Type	Description	Required?
value	string	The text of the timestamp.	yes
format	string	The strptime format.	yes
timezone	string	The TZ database format.	no

parse_tokens

fallible pure

Parses the value in token format. A token can be a word surrounded by whitespace, text delimited by double quotes ("..."), or text delimited by square brackets ([...]).

Function spec

parse_tokens(value: <string>)
  :: <array>, <error>

Argument	Type	Description	Default	Required?
value	string	The string to tokenize.		yes

Notices

All token values are returned as strings.

Errors

The parse_tokens function is fallible, requiring error handling if value is not a properly formatted tokenized string.

Examples

Parse tokens

Source

parse_tokens("A sentence \"with \\\"a\\\" sentence inside\" and [some brackets]")

Return

["A", "sentence", "with \\\"a\\\" sentence inside", "and", "some brackets"]

parse_url

fallible pure

Parses the value in URL format.

Function spec

parse_url(value: <string>, [default_known_ports: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The text of the URL.		yes
default_known_ports	boolean	If true, the port is populated from well-known ports if not specified for certain schemes (http, https, ws, wss, ftp).		no

Examples

Parse URL with internationalized domain name

Source

parse_url!("https://www.café.com")

Return

{
  "fragment": null,
  "host": "www.xn--caf-dma.com",
  "password": "",
  "path": "/",
  "port": null,
  "query": {},
  "scheme": "https",
  "username": ""
}

parse_user_agent

infallible pure

Parses the value as a user agent string, which has a loosely defined format so this parser only provides best effort guarantee.

Function spec

parse_user_agent(value: <string>, [mode: <string>])
  :: <object>

Argument	Type	Description	Default	Required?
value	string	The string to parse.		yes
mode	string	Determines performance and reliability characteristics.	fast	no

Notices

All values are returned as strings or as null. Different modes return different schema.

Examples

Fast mode

Source

parse_user_agent("Mozilla Firefox 1.0.1 Mozilla/5.0 (X11; U; Linux i686; de-DE; rv:1.7.6) Gecko/20050223 Firefox/1.0.1")

Return

{
  "browser": {"family": "Firefox", "version": "1.0.1"},
  "device": {"category": "pc"},
  "os": {"family": "Linux", "version": null}
}

parse_xml

fallible pure

Parses the value as XML.

Function spec

parse_xml(value: <string>, [include_attr: <boolean>, attr_prefix: <string>, text_key: <string>, always_use_text_key: <boolean>, parse_bool: <boolean>, parse_null: <boolean>, parse_number: <boolean>])
  :: <object>, <error>

Argument	Type	Description	Default	Required?
value	string	The string representation of the XML document to parse.		yes
include_attr	boolean	Include XML tag attributes in the returned object.	true	no
attr_prefix	string	String prefix to use for XML tag attribute keys.	@	no

Notices

Valid XML must contain exactly one root node.

Errors

The parse_xml function is fallible, requiring error handling if value is not a valid XML document.

Examples

Parse XML

Source

value = s'<book category="CHILDREN"><title lang="en">Harry Potter</title><author>J K. Rowling</author><year>2005</year></book>';
parse_xml!(value, text_key: "value", parse_number: false)

Return

{
  "book": {
    "@category": "CHILDREN",
    "author": "J K. Rowling",
    "title": {"@lang": "en", "value": "Harry Potter"},
    "year": "2005"
  }
}

Enumerate functions

compact

infallible pure

Compacts the value by removing empty values, where empty values are defined using the available parameters.

Function spec

compact(value: <object | array>, [recursive: <boolean>, null: <boolean>, string: <boolean>, object: <boolean>, array: <boolean>, nullish: <boolean>])
  :: <object | array>

Argument	Type	Description	Default	Required?
value	object, array	The object or array to compact.		yes
recursive	boolean	Whether the compaction be recursive.	true	no
null	boolean	Whether null should be treated as an empty value.	true	no
string	boolean	Whether an empty string should be treated as an empty value.	true	no
object	boolean	Whether an empty object should be treated as an empty value.	true	no
array	boolean	Whether an empty array should be treated as an empty value.	true	no
nullish	boolean	Tests whether the value is "nullish" as defined by the is_nullish function.	false	no

Examples

Compact an object with default parameters

Source

compact({"field1": 1, "field2": "", "field3": [], "field4": null})

Return

{"field1": 1}

Compact an array with default parameters

Source

compact(["foo", "bar", "", null, [], "buzz"])

Return

["foo", "bar", "buzz"]

Compact an array using nullish

Source

compact(["-", "   ", "\n", null, true], nullish: true)

Return

[true]

Compact a complex object with default parameters

Source

compact({"a": {}, "b": null, "c": [null], "d": "", "e": "-", "f": true})

Return

{"e": "-", "f": true}

Compact a complex object using null: false

Source

compact({"a": {}, "b": null, "c": [null], "d": "", "e": "-", "f": true}, null: false)

Return

{"b": null, "c": [null], "e": "-", "f": true}

filter

infallible pure

Filter elements from a collection.

This function currently does not support recursive iteration.

The function uses the function closure syntax to allow reading the key-value or index-value combination for each item in the collection.

The same scoping rules apply to closure blocks as they do for regular blocks. This means that any variable defined in parent scopes is accessible, and mutations to those variables are preserved, but any new variables instantiated in the closure block are unavailable outside of the block.

Function spec

filter(value: <object | array>)
  :: <object | array>

Argument	Type	Description	Default	Required?
value	object, array	The array or object to filter.		yes

Examples

Filter elements

Source

. = {"tags": ["foo", "bar", "foo", "baz"]}
filter(array(.tags)) -> |_index, value| {
    value != "foo"
}

Return

["bar", "baz"]

Filter object

Source

filter({"a": 1, "b": 2}) -> |key, _value| { key == "a" }

Return

{"a": 1}

Filter array

Source

filter([1, 2]) -> |_index, value| { value < 2 }

Return

[1]

flatten

infallible pure

Flattens the value into a single-level representation.

Function spec

flatten(value: <object | array>, [separator: <string>])
  :: <object | array>

Argument	Type	Description	Default	Required?
value	object, array	The array or object to flatten.		yes
separator	string	The separator to join nested keys.	.	no

Examples

Flatten array

Source

flatten([1, [2, 3, 4], [5, [6, 7], 8], 9])

Return

[1, 2, 3, 4, 5, 6, 7, 8, 9]

Flatten object

Source

flatten({
    "parent1": {
        "child1": 1,
        "child2": 2
    },
    "parent2": {
        "child3": 3
    }
})

Return

{
  "parent1.child1": 1,
  "parent1.child2": 2,
  "parent2.child3": 3
}

Flatten object with custom separator

Source

flatten({"foo": {"bar": true}}, "_")

Return

{"foo_bar": true}

for_each

infallible pure

Iterate over a collection.

This function currently does not support recursive iteration.

The function uses the "function closure syntax" to allow reading the key/value or index/value combination for each item in the collection.

Function spec

for_each(value: <object | array>)
  :: <null>

Argument	Type	Description	Default	Required?
value	object, array	The array or object to iterate.		yes

Examples

Tally elements

Source

.tags = ["foo", "bar", "foo", "baz"]
tally = {}
for_each(array(.tags)) -> |_index, value| {
    count = int(get!(tally, [value])) ?? 0
    tally = set!(tally, [value], count + 1)
}
tally

Return

{"bar": 1, "baz": 1, "foo": 2}

Iterate over an object

Source

count = 0
for_each({"a": 1, "b": 2}) -> |_key, value| {
    count = count + value
}
count

Return

Iterate over an array

Source

count = 0
for_each([1, 2, 3]) -> |index, value| {
    count = count + index + value
}
count

Return

includes

infallible pure

Determines whether the value array includes the specified item.

Function spec

includes(value: <array>, item: <any>)
  :: <boolean>

Argument	Type	Description	Default	Required?
value	array	The array.		yes
item	any	The item to check.		yes

Examples

Array includes

Source

includes(["apple", "orange", "banana"], "banana")

Return

true

Includes boolean

Source

includes([1, true], true)

Return

true

Doesn't include

Source

includes(["foo", "bar"], "baz")

Return

false

keys

infallible pure

Returns the keys from the object passed into the function.

Function spec

keys(value: <object>)
  :: <array>

Argument	Type	Description	Default	Required?
value	object	The object to extract keys from.		yes

Examples

Get keys from the object

Source

keys({"key1": "val1", "key2": "val2"})

Return

["key1", "key2"]

map_keys

infallible pure

Map the keys within an object.

If recursive is enabled, the function iterates into nested objects, using the following rules:

Iteration starts at the root.
For every nested object type:
- First return the key of the object type itself.
- Then recurse into the object, and loop back to item (1) in this list.
Any mutation done on a nested object before recursing into it, are preserved.
For every nested array type:
- First return the key of the array type itself.
- Then find all objects within the array, and apply item (2) to each individual object.

The above rules mean that map_keys with recursive enabled finds all keys in the target, regardless of whether nested objects are nested inside arrays.

The function uses the function closure syntax to allow reading the key for each item in the object.

Function spec

map_keys(value: <object>, [recursive: <boolean>])
  :: <object>

Argument	Type	Description	Default	Required?
value	object	The object to iterate.		yes
recursive	boolean	Whether to recursively iterate the collection.	false	no

Examples

Upcase keys

Source

. = {
    "foo": "foo",
    "bar": "bar",
    "baz": {"nested key": "val"}
}
map_keys(.) -> |key| { upcase(key) }

Return

{
  "BAR": "bar",
  "BAZ": {"nested key": "val"},
  "FOO": "foo"
}

De-dot keys

Source

. = {
    "labels": {
        "app.kubernetes.io/name": "mysql"
    }
}
map_keys(., recursive: true) -> |key| { replace(key, ".", "_") }

Return

{
  "labels": {
    "app_kubernetes_io/name": "mysql"
  }
}

Recursively map object keys

Source

val = {
    "a": 1,
    "b": [{"c": 2}, {"d": 3}],
    "e": {"f": 4}
}
map_keys(val, recursive: true) -> |key| { upcase(key) }

Return

{
  "A": 1,
  "B": [{"C": 2}, {"D": 3}],
  "E": {"F": 4}
}

map_values

infallible pure

Map the values within a collection.

If recursive is enabled, the function iterates into nested collections, using the following rules:

Iteration starts at the root.
For every nested collection type:
- First return the collection type itself.
- Then recurse into the collection, and loop back to item (1) in the list.
Any mutation done on a collection before recursing into it, are preserved.

The function uses the function closure syntax to allow mutating the value for each item in the collection.

Function spec

map_values(value: <object | array>, [recursive: <boolean>])
  :: <object | array>

Argument	Type	Description	Default	Required?
value	object, array	The object or array to iterate.		yes
recursive	boolean	Whether to recursively iterate the collection.	false	no

Examples

Upcase values

Source

. = {
    "foo": "foo",
    "bar": "bar"
}
map_values(.) -> |value| { upcase(value) }

Return

{"bar": "BAR", "foo": "FOO"}

Recursively map object values

Source

val = {
    "a": 1,
    "b": [{"c": 2}, {"d": 3}],
    "e": {"f": 4}
}
map_values(val, recursive: true) -> |value| {
    if is_integer(value) { int!(value) + 1 } else { value }
}

Return

{
  "a": 2,
  "b": [{"c": 3}, {"d": 4}],
  "e": {"f": 5}
}

match_array

infallible pure

Determines whether the elements in the value array matches the pattern. By default, it checks that at least one element matches, but can be set to determine if all the elements match.

Function spec

match_array(value: <array>, pattern: <regex>, [all: <boolean>])
  :: <boolean>

Argument	Type	Description	Default	Required?
value	array	The array.		yes
pattern	regex	The regular expression pattern to match against.		yes
all	boolean	Whether to match on all elements of `value`.	false	no

Examples

Match at least one element

Source

match_array(["foobar", "bazqux"], r'foo')

Return

true

Match all elements

Source

match_array(["foo", "foobar", "barfoo"], r'foo', all: true)

Return

true

No matches

Source

match_array(["bazqux", "xyz"], r'foo')

Return

false

Not all elements match

Source

match_array(["foo", "foobar", "baz"], r'foo', all: true)

Return

false

strlen

infallible pure

Returns the number of UTF-8 characters in value. This differs from length which counts the number of bytes of a string.

Note: This is the count of Unicode scalar values which can sometimes differ from Unicode code points.

Function spec

strlen(value: <string>)
  :: <integer>

Argument	Type	Description	Default	Required?
value	string	The string.		yes

Examples

Count Unicode scalar values

Source

strlen("ñandú")

Return

tally

infallible pure

Counts the occurrences of each string value in the provided array and returns an object with the counts.

Function spec

tally(value: <array>)
  :: <object>

Argument	Type	Description	Default	Required?
value	array	The array of strings to count occurrences for.		yes

Examples

tally

Source

tally!(["foo", "bar", "foo", "baz"])

Return

{"bar": 1, "baz": 1, "foo": 2}

tally_value

infallible pure

Counts the number of times a specific value appears in the provided array.

Function spec

tally_value(array: <array>, value: <any>)
  :: <integer>

Argument	Type	Description	Default	Required?
array	array	The array to search through.		yes
value	any	The value to count occurrences of in the array.		yes

Examples

count matching values

Source

tally_value(["foo", "bar", "foo", "baz"], "foo")

Return

unflatten

infallible pure

Unflattens the value into a nested representation.

Function spec

unflatten(value: <object>, [separator: <string>, recursive: <boolean>])
  :: <object>

Argument	Type	Description	Default	Required?
value	object	The array or object to unflatten.		yes
separator	string	The separator to split flattened keys.	.	no
recursive	boolean	Whether to recursively unflatten the object values.	true	no

Examples

Unflatten

Source

unflatten({
    "foo.bar.baz": true,
    "foo.bar.qux": false,
    "foo.quux": 42
})

Return

{
  "foo": {
    "bar": {"baz": true, "qux": false},
    "quux": 42
  }
}

Unflatten recursively

Source

unflatten({
    "flattened.parent": {
        "foo.bar": true,
        "foo.baz": false
    }
})

Return

{
  "flattened": {
    "parent": {
      "foo": {"bar": true, "baz": false}
    }
  }
}

Unflatten non-recursively

Source

unflatten({
    "flattened.parent": {
        "foo.bar": true,
        "foo.baz": false
    }
}, recursive: false)

Return

{
  "flattened": {
    "parent": {
      "foo.bar": true,
      "foo.baz": false
    }
  }
}

Ignore inconsistent keys values

Source

unflatten({
    "a": 3,
    "a.b": 2,
    "a.c": 4
})

Return

{"a": {"b": 2, "c": 4}}

Unflatten with custom separator

Source

unflatten({"foo_bar": true}, "_")

Return

{"foo": {"bar": true}}

unique

infallible pure

Returns the unique values for an array.

The first occurrence of each element is kept.

Function spec

unique(value: <array>)
  :: <array>

Argument	Type	Description	Default	Required?
value	array	The array to return unique elements from.		yes

Examples

Unique

Source

unique(["foo", "bar", "foo", "baz"])

Return

["foo", "bar", "baz"]

values

infallible pure

Returns the values from the object passed into the function.

Function spec

values(value: <object>)
  :: <array>

Argument	Type	Description	Default	Required?
value	object	The object to extract values from.		yes

Examples

Get values from the object

Source

values({"key1": "val1", "key2": "val2"})

Return

["val1", "val2"]

Get values from a complex object

Source

values({"key1": "val1", "key2": [1, 2, 3], "key3": {"foo": "bar"}})

Return

["val1", [1, 2, 3], {"foo": "bar"}]

Chỉnh sửa trang nàyhoặcBáo cáo sự cố

pfSense

Overview