Parser Example
FortiGate
FortiGate
Overview
Parser xử lý FortiGate logs trong JSON Wrapped Format - logs được ship qua Filebeat/Fluentd/Logstash với message field chứa key-value pairs theo tài liệu chính thức Fortinet 7.0.12.
Input Format:
{"message":"date=2019-05-10 time=11:37:47 logid=\"0000000013\" type=\"traffic\"..."}
Log Types:
- Traffic logs: type="traffic", subtype="forward/local/multicast/sniffer"
- UTM logs: type="utm", subtype="virus/webfilter/app-ctrl/ips/anomaly/dlp/dns/ssh/ssl"
- Event logs: type="event", subtype="system/user/vpn/router/sdwan/ha/wireless"
Message Content: key=value key="quoted value" key=number
Sample Logs (JSON Wrapped Format)
1. Traffic - Forward Accept
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-10 time=11:37:47 logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"notice\" vd=\"vdom1\" eventtime=1557513467 srcip=10.1.100.11 srcport=58012 srcintf=\"port12\" srcintfrole=\"lan\" dstip=23.59.154.35 dstport=80 dstintf=\"port11\" dstintfrole=\"wan\" poluuid=\"ccb269e0-5735-51e9-a218-a397dd08b7eb\" sessionid=105048 proto=6 action=\"accept\" policyid=1 policytype=\"policy\" service=\"HTTP\" dstcountry=\"Canada\" srccountry=\"Reserved\" duration=116 sentbyte=1188 rcvdbyte=1224 sentpkt=17 rcvdpkt=16 hostname=\"example.com\" user=\"alice\" group=\"users\""}
2. Traffic - Forward Deny
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-10 time=14:25:30 logid=\"0000000013\" type=\"traffic\" subtype=\"forward\" level=\"warning\" vd=\"vdom1\" eventtime=1557523530 srcip=192.168.1.200 srcport=45678 srcintf=\"port12\" srcintfrole=\"lan\" dstip=10.0.0.50 dstport=22 dstintf=\"port11\" dstintfrole=\"wan\" sessionid=105050 proto=6 action=\"deny\" policyid=2 policytype=\"policy\" service=\"SSH\" dstcountry=\"Reserved\" srccountry=\"Reserved\" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=1 rcvdpkt=0"}
3. UTM - IPS (Intrusion Prevention)
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-15 time=17:56:41 logid=\"0419016384\" type=\"utm\" subtype=\"ips\" eventtype=\"signature\" level=\"alert\" vd=\"root\" eventtime=1557968201 severity=\"critical\" srcip=10.1.100.22 srccountry=\"Reserved\" dstip=172.16.200.55 srcintf=\"port10\" srcintfrole=\"lan\" dstintf=\"port9\" dstintfrole=\"wan\" sessionid=4017 action=\"dropped\" proto=6 service=\"HTTP\" policyid=1 attack=\"Adobe.Flash.newfunction.Handling.Code.Execution\" srcport=46810 dstport=80 hostname=\"172.16.200.55\" url=\"/ips/sig1.pdf\" direction=\"incoming\" attackid=23305 profile=\"block-critical-ips\" msg=\"IPS attack detected\" crscore=50"}
4. UTM - Antivirus
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-13 time=11:45:03 logid=\"0211008192\" type=\"utm\" subtype=\"virus\" eventtype=\"infected\" level=\"warning\" vd=\"vdom1\" eventtime=1557773103 msg=\"File is infected.\" action=\"blocked\" service=\"HTTP\" sessionid=359260 srcip=10.1.100.11 dstip=172.16.200.55 srcport=60446 dstport=80 srcintf=\"port12\" srcintfrole=\"lan\" dstintf=\"port11\" dstintfrole=\"wan\" policyid=4 proto=6 direction=\"incoming\" filename=\"eicar.com\" virus=\"EICAR_TEST_FILE\" dtype=\"Virus\" virusid=2172 url=\"http://172.16.200.55/virus/eicar.com\" profile=\"g-default\" crscore=50"}
5. UTM - Web Filter
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-13 time=16:29:45 logid=\"0316013056\" type=\"utm\" subtype=\"webfilter\" eventtype=\"ftgd_blk\" level=\"warning\" vd=\"vdom1\" eventtime=1557790184 policyid=1 sessionid=381780 srcip=10.1.100.11 srcport=44258 srcintf=\"port12\" srcintfrole=\"lan\" dstip=185.244.31.158 dstport=80 dstintf=\"port11\" dstintfrole=\"wan\" proto=6 service=\"HTTP\" hostname=\"morrishittu.ddns.net\" profile=\"test-webfilter\" action=\"blocked\" reqtype=\"direct\" url=\"/\" sentbyte=84 rcvdbyte=0 direction=\"outgoing\" msg=\"URL belongs to a denied category in policy\" method=\"domain\" cat=26 catdesc=\"Malicious Websites\" crscore=30"}
6. UTM - Application Control
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-15 time=18:03:36 logid=\"1059028704\" type=\"utm\" subtype=\"app-ctrl\" eventtype=\"app-ctrl-all\" level=\"information\" vd=\"root\" eventtime=1557968615 appid=40568 srcip=10.1.100.22 dstip=195.8.215.136 srcport=50798 dstport=443 srcintf=\"port10\" srcintfrole=\"lan\" dstintf=\"port9\" dstintfrole=\"wan\" proto=6 service=\"HTTPS\" direction=\"outgoing\" policyid=1 sessionid=4414 applist=\"block-social.media\" appcat=\"Web.Client\" app=\"HTTPS.BROWSER\" action=\"pass\" hostname=\"www.dailymotion.com\" url=\"/\" msg=\"Web.Client: HTTPS.BROWSER,\" apprisk=\"medium\""}
7. UTM - DNS Query
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-15 time=15:05:49 logid=\"1501054802\" type=\"utm\" subtype=\"dns\" eventtype=\"dns-response\" level=\"notice\" vd=\"vdom1\" eventtime=1557957949 policyid=1 sessionid=6887 srcip=10.1.100.22 srcport=50002 srcintf=\"port12\" srcintfrole=\"lan\" dstip=172.16.100.100 dstport=53 dstintf=\"port11\" dstintfrole=\"wan\" proto=17 profile=\"dnsfilter_fgd\" xid=57945 qname=\"changelogs.ubuntu.com\" qtype=\"AAAA\" qtypeval=28 qclass=\"IN\" ipaddr=\"2001:67c:1560:8008::11\" msg=\"Domain is monitored\" action=\"pass\" cat=52 catdesc=\"Information Technology\""}
8. Event - System Authentication Success
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-13 time=11:20:54 logid=\"0100032001\" type=\"event\" subtype=\"system\" level=\"information\" vd=\"vdom1\" eventtime=1557771654 logdesc=\"Admin login successful\" user=\"admin\" ui=\"ssh(172.16.200.254)\" method=\"ssh\" srcip=172.16.200.254 dstip=172.16.200.2 action=\"login\" status=\"success\" reason=\"none\" profile=\"super_admin\" msg=\"Administrator admin logged in successfully from ssh(172.16.200.254)\""}
9. Event - User Authentication Failed
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-13 time=15:55:56 logid=\"0102043009\" type=\"event\" subtype=\"user\" level=\"warning\" vd=\"root\" eventtime=1557788156 logdesc=\"Authentication failed\" srcip=10.1.100.11 dstip=172.16.200.55 policyid=1 interface=\"port10\" user=\"bob\" group=\"local-group1\" authproto=\"TELNET(10.1.100.11)\" action=\"authentication\" status=\"failure\" reason=\"invalid_password\" msg=\"User bob failed in authentication\""}
10. UTM - Anomaly (DoS)
{"timestamp":"2026-01-07T07:55:06.703Z","message":"date=2019-05-13 time=17:05:59 logid=\"0720018433\" type=\"utm\" subtype=\"anomaly\" eventtype=\"anomaly\" level=\"alert\" vd=\"vdom1\" eventtime=1557792359 severity=\"critical\" srcip=10.1.100.11 srccountry=\"Reserved\" dstip=172.16.200.55 srcintf=\"port12\" srcintfrole=\"undefined\" sessionid=0 action=\"clear_session\" proto=1 service=\"PING\" count=1 attack=\"icmp_flood\" attackid=16777316 policyid=1 policytype=\"DoS-policy\" msg=\"anomaly: icmp_flood, 51 > threshold 50\" crscore=50"}
Parser Configuration
#regex
#conditional
log_message = ""
if (msg, err = to_string(.message); err == null) {
log_message = msg
}
event_timestamp = ""
log_date = ""
log_time = ""
event_module = "fortigate"
log_id = ""
log_type = ""
log_subtype = ""
event_category = ""
event_action = ""
event_outcome = ""
log_level = "info"
source_ip = ""
source_port = ""
source_interface = ""
destination_ip = ""
destination_port = ""
destination_interface = ""
network_protocol = ""
network_transport = ""
user_name = ""
group_name = ""
policy_id = ""
policy_uuid = ""
session_id = ""
action = ""
threat_name = ""
threat_category = ""
attack_name = ""
attack_id = ""
url_full = ""
url_domain = ""
file_name = ""
server_address = ""
client_address = ""
if (date_m, err = parse_regex(log_message, r'date=(?P<date>\d{4}-\d{2}-\d{2})'); err == null) {
log_date = to_string(date_m.date)
}
if (time_m, err = parse_regex(log_message, r'time=(?P<time>\d{2}:\d{2}:\d{2})'); err == null) {
log_time = to_string(time_m.time)
}
if (logid_m, err = parse_regex(log_message, r'logid="(?P<id>[^"]+)"'); err == null) {
log_id = to_string(logid_m.id)
}
if (type_m, err = parse_regex(log_message, r'\stype="(?P<type>[^"]+)"'); err == null) {
log_type = to_string(type_m.type)
}
if (subtype_m, err = parse_regex(log_message, r'subtype="(?P<subtype>[^"]+)"'); err == null) {
log_subtype = to_string(subtype_m.subtype)
}
if (level_m, err = parse_regex(log_message, r'level="(?P<level>[^"]+)"'); err == null) {
log_level = to_string(level_m.level)
}
if (srcip_m, err = parse_regex(log_message, r'srcip=(?P<ip>[0-9.]+)'); err == null) {
source_ip = to_string(srcip_m.ip)
}
if (srcport_m, err = parse_regex(log_message, r'srcport=(?P<port>\d+)'); err == null) {
source_port = to_string(srcport_m.port)
}
if (srcintf_m, err = parse_regex(log_message, r'srcintf="(?P<intf>[^"]+)"'); err == null) {
source_interface = to_string(srcintf_m.intf)
}
if (dstip_m, err = parse_regex(log_message, r'dstip=(?P<ip>[0-9.]+)'); err == null) {
destination_ip = to_string(dstip_m.ip)
}
if (dstport_m, err = parse_regex(log_message, r'dstport=(?P<port>\d+)'); err == null) {
destination_port = to_string(dstport_m.port)
}
if (dstintf_m, err = parse_regex(log_message, r'dstintf="(?P<intf>[^"]+)"'); err == null) {
destination_interface = to_string(dstintf_m.intf)
}
if (proto_m, err = parse_regex(log_message, r'\sproto=(?P<proto>\d+)'); err == null) {
proto_num = to_string(proto_m.proto)
if proto_num == "6" { network_transport = "tcp" }
if proto_num == "17" { network_transport = "udp" }
if proto_num == "1" { network_transport = "icmp" }
}
if (service_m, err = parse_regex(log_message, r'service="(?P<service>[^"]+)"'); err == null) {
network_protocol = downcase(to_string(service_m.service))
}
if (policyid_m, err = parse_regex(log_message, r'policyid=(?P<id>\d+)'); err == null) {
policy_id = to_string(policyid_m.id)
}
if (poluuid_m, err = parse_regex(log_message, r'poluuid="(?P<uuid>[^"]+)"'); err == null) {
policy_uuid = to_string(poluuid_m.uuid)
}
if (sessionid_m, err = parse_regex(log_message, r'sessionid=(?P<id>\d+)'); err == null) {
session_id = to_string(sessionid_m.id)
}
if (action_m, err = parse_regex(log_message, r'action="(?P<action>[^"]+)"'); err == null) {
action = to_string(action_m.action)
}
if (user_m, err = parse_regex(log_message, r'\suser="(?P<user>[^"]+)"'); err == null) {
user_name = to_string(user_m.user)
}
if (group_m, err = parse_regex(log_message, r'group="(?P<group>[^"]+)"'); err == null) {
group_name = to_string(group_m.group)
}
if (hostname_m, err = parse_regex(log_message, r'hostname="(?P<host>[^"]+)"'); err == null) {
url_domain = to_string(hostname_m.host)
server_address = to_string(hostname_m.host)
}
if (url_m, err = parse_regex(log_message, r'\surl="(?P<url>[^"]+)"'); err == null) {
url_full = to_string(url_m.url)
}
if (filename_m, err = parse_regex(log_message, r'filename="(?P<file>[^"]+)"'); err == null) {
file_name = to_string(filename_m.file)
}
if (attack_m, err = parse_regex(log_message, r'attack="(?P<attack>[^"]+)"'); err == null) {
attack_name = to_string(attack_m.attack)
threat_name = attack_name
}
if (attackid_m, err = parse_regex(log_message, r'attackid=(?P<id>\d+)'); err == null) {
attack_id = to_string(attackid_m.id)
}
if (virus_m, err = parse_regex(log_message, r'virus="(?P<virus>[^"]+)"'); err == null) {
threat_name = to_string(virus_m.virus)
}
if (catdesc_m, err = parse_regex(log_message, r'catdesc="(?P<cat>[^"]+)"'); err == null) {
threat_category = to_string(catdesc_m.cat)
}
log_type_str = to_string(log_type)
log_subtype_str = to_string(log_subtype)
action_str = to_string(action)
if log_type_str == "traffic" {
event_category = "network"
event_module = "fortigate.traffic"
if action_str == "accept" {
event_action = "allowed"
event_outcome = "success"
}
if action_str == "deny" {
event_action = "denied"
event_outcome = "failure"
}
if action_str == "close" {
event_action = "connection_closed"
event_outcome = "success"
}
}
if log_type_str == "utm" {
if log_subtype_str == "ips" {
event_category = "intrusion_detection"
event_module = "fortigate.ips"
if action_str == "dropped" {
event_action = "blocked"
event_outcome = "success"
}
if action_str == "detected" {
event_action = "detected"
event_outcome = "success"
}
}
if log_subtype_str == "virus" {
event_category = "malware"
event_module = "fortigate.antivirus"
if action_str == "blocked" {
event_action = "blocked"
event_outcome = "success"
}
if action_str == "detected" {
event_action = "detected"
event_outcome = "success"
}
}
if log_subtype_str == "webfilter" {
event_category = "web"
event_module = "fortigate.webfilter"
if action_str == "blocked" {
event_action = "blocked"
event_outcome = "success"
}
if action_str == "passthrough" {
event_action = "allowed"
event_outcome = "success"
}
}
if log_subtype_str == "app-ctrl" {
event_category = "network"
event_module = "fortigate.application"
if action_str == "blocked" {
event_action = "blocked"
event_outcome = "success"
}
if action_str == "pass" {
event_action = "allowed"
event_outcome = "success"
}
}
if log_subtype_str == "dns" {
event_category = "network"
event_module = "fortigate.dns"
if action_str == "pass" {
event_action = "allowed"
event_outcome = "success"
}
if action_str == "blocked" {
event_action = "blocked"
event_outcome = "success"
}
}
if log_subtype_str == "anomaly" {
event_category = "intrusion_detection"
event_module = "fortigate.anomaly"
threat_name = attack_name
if action_str == "clear_session" {
event_action = "blocked"
event_outcome = "success"
}
}
}
if log_type_str == "event" {
if log_subtype_str == "system" {
event_category = "authentication"
event_module = "fortigate.event"
if contains(log_message, "login") {
if contains(log_message, "success") {
event_action = "logon_success"
event_outcome = "success"
}
if contains(log_message, "failed") {
event_action = "logon_failed"
event_outcome = "failure"
}
}
}
if log_subtype_str == "user" {
event_category = "authentication"
event_module = "fortigate.event"
if contains(log_message, "authentication") {
if contains(log_message, "success") {
event_action = "logon_success"
event_outcome = "success"
}
if contains(log_message, "failure") {
event_action = "logon_failed"
event_outcome = "failure"
}
}
}
if log_subtype_str == "vpn" {
event_category = "network"
event_module = "fortigate.vpn"
event_action = "vpn_event"
event_outcome = "success"
}
if log_subtype_str == "router" {
event_category = "network"
event_module = "fortigate.router"
event_action = "routing_event"
event_outcome = "success"
}
}
if source_ip != "" { client_address = source_ip }
if event_category == "" { event_category = "network" }
if event_action == "" { event_action = "info" }
if event_outcome == "" { event_outcome = "unknown" }
#normalize
timestamp: format_timestamp!(parse_timestamp!(.timestamp, "%Y-%m-%dT%H:%M:%S.%3fZ"), "%Y-%m-%d %H:%M:%S")
event.module: event_module
event.category: event_category
event.action: event_action
event.outcome: event_outcome
log.level: log_level
message: log_message
source.ip: source_ip
source.port: source_port
destination.ip: destination_ip
destination.port: destination_port
network.transport: network_transport
network.protocol: network_protocol
user.name: user_name
group.name: group_name
url.full: url_full
url.domain: url_domain
file.name: file_name
server.address: server_address
client.address: client_address
threat.name: threat_name
labels.log_id: log_id
labels.log_type: log_type
labels.log_subtype: log_subtype
labels.policy_id: policy_id
labels.policy_uuid: policy_uuid
labels.session_id: session_id
labels.action: action
labels.source_interface: source_interface
labels.destination_interface: destination_interface
labels.threat_category: threat_category
labels.attack_name: attack_name
labels.attack_id: attack_id
Event Categories
| Log Type | Subtype | Action | Event Category | Event Action | Event Outcome |
|---|---|---|---|---|---|
| traffic | forward | accept | network | allowed | success |
| traffic | forward | deny | network | denied | failure |
| traffic | forward | close | network | connection_closed | success |
| utm | ips | dropped | intrusion_detection | blocked | success |
| utm | ips | detected | intrusion_detection | detected | success |
| utm | virus | blocked | malware | blocked | success |
| utm | webfilter | blocked | web | blocked | success |
| utm | app-ctrl | blocked | network | blocked | success |
| utm | app-ctrl | pass | network | allowed | success |
| utm | dns | pass | network | allowed | success |
| utm | anomaly | clear_session | intrusion_detection | blocked | success |
| event | system | login success | authentication | logon_success | success |
| event | system | login failed | authentication | logon_failed | failure |
| event | user | authentication success | authentication | logon_success | success |
| event | user | authentication failure | authentication | logon_failed | failure |
Field Mapping
| FortiGate Field | ECS Field | Field Set |
|---|---|---|
| date + time | timestamp | Base |
| srcip | source.ip, client.address | Source |
| srcport | source.port | Source |
| srcintf | labels.source_interface | Labels |
| dstip | destination.ip | Destination |
| dstport | destination.port | Destination |
| dstintf | labels.destination_interface | Labels |
| proto | network.transport | Network |
| service | network.protocol | Network |
| user | user.name | User |
| group | group.name | Group |
| hostname | url.domain, server.address | URL, Server |
| url | url.full | URL |
| filename | file.name | File |
| attack | threat.name | Threat |
| virus | threat.name | Threat |
| catdesc | labels.threat_category | Labels |
| policyid | labels.policy_id | Labels |
| sessionid | labels.session_id | Labels |
| action | labels.action | Labels |
Notes
Input Format: JSON object với field .message chứa FortiGate key-value pairs.
Parsing Strategy:
- Extract
.messagefield từ JSON wrapper - Parse key-value pairs với regex
key="value"hoặckey=number - Combine date + time thành timestamp
- Map proto numbers: 6=tcp, 17=udp, 1=icmp
- Normalize log.level và event fields theo ECS
VRL Functions Used:
to_string()- Type coercionparse_regex()- Key-value extractioncontains()- String matchingdowncase()- Lowercase conversion
Test Results: Xem TEST_RESULTS.md cho chi tiết test cases.