Sekoia.io forwarder logs

Overview

• Vendor: Sekoia
• Plan: Defend Core & Defend Prime
• Detection based on: Audit

• Supported application or feature: Sekoia.io forwarder logs collect all statictics coming from Sekoia forwarder instances. It helps to monitor the forwarder health:

  • resource usage
  • queue size
  • number of messages received by the forwarder
  • number of messages sent by the forwarder

Configure

To monitor forwarder health, create a new intake Sekoia.io forwarer logs in your community. Once the intake is enabled, please follow this documentation in order to activate metrics on the forwarder side. You can find also details about the generated metrics

Raw Events Samples

In this section, you will find examples of raw logs as generated natively by the source. These examples are provided to help integrators understand the data format before ingestion into Sekoia.io. It is crucial for setting up the correct parsing stages and ensuring that all relevant information is captured.

action_logda_queue_loginput_tcpinput_udpmemory_queue_logoutput_logstats

{
    "name": "output-techno1",
    "origin": "core.action",
    "processed": 0,
    "failed": 0,
    "suspended": 0,
    "suspended.duration": 0,
    "resumed": 0
}

{
    "name": "ruleset-my-techno1[DA]",
    "origin": "core.queue",
    "size": 0,
    "enqueued": 0,
    "full": 0,
    "discarded.full": 0,
    "discarded.nf": 0,
    "maxqsize": 0
}

{
    "name": "input-techno1(20516)",
    "origin": "imtcp",
    "submitted": 0
}

{
    "name": "input-my-techno2(*/20517/IPv4)",
    "origin": "imudp",
    "submitted": 0,
    "disallowed": 0
}

{
    "name": "ruleset-techno1",
    "origin": "core.queue",
    "size": 0,
    "enqueued": 0,
    "full": 0,
    "discarded.full": 0,
    "discarded.nf": 0,
    "maxqsize": 0
}

{
    "name": "TCP-intake.sekoia.io-10514",
    "origin": "omfwd",
    "bytes.sent": 0
}

{
    "name": "resource-usage",
    "origin": "impstats",
    "utime": 134625,
    "stime": 20397,
    "maxrss": 13888,
    "minflt": 1510,
    "majflt": 0,
    "inblock": 0,
    "oublock": 272,
    "nvcsw": 347,
    "nivcsw": 31,
    "openfiles": 15
}

Detection section

The following section provides information for those who wish to learn more about the detection capabilities enabled by collecting this intake. It includes details about the built-in rule catalog, event categories, and ECS fields extracted from raw events. This is essential for users aiming to create custom detection rules, perform hunting activities, or pivot in the events page.

No related built-in rules was found. This message is automatically generated.

Event Categories

The following table lists the data source offered by this integration.

Data Source	Description
Process monitoring	Sekoia forwarder logs are collected: rsyslog resource usage, queue size, number of messages handled, ..

Transformed Events Samples after Ingestion

This section demonstrates how the raw logs will be transformed by our parsers. It shows the extracted fields that will be available for use in the built-in detection rules and hunting activities in the events page. Understanding these transformations is essential for analysts to create effective detection mechanisms with custom detection rules and to leverage the full potential of the collected data.

action_log.jsonda_queue_log.jsoninput_tcp.jsoninput_udp.jsonmemory_queue_log.jsonoutput_log.jsonstats.json

{
    "message": "{\"name\":\"output-techno1\",\"origin\":\"core.action\",\"processed\": 0,\"failed\": 0,\"suspended\": 0, \"suspended.duration\": 0, \"resumed\": 0}",
    "sekoiaio": {
        "forwarder": {
            "action": {
                "failed": 0,
                "processed": 0,
                "resumed": 0,
                "suspended": 0
            },
            "intake": {
                "name": "techno1"
            },
            "module": {
                "name": "output-techno1",
                "type": "core.action"
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"ruleset-my-techno1[DA]\", \"origin\": \"core.queue\", \"size\": 0, \"enqueued\": 0, \"full\": 0, \"discarded.full\": 0, \"discarded.nf\": 0, \"maxqsize\": 0 }",
    "sekoiaio": {
        "forwarder": {
            "intake": {
                "name": "my-techno1"
            },
            "module": {
                "name": "ruleset-my-techno1[DA]",
                "type": "core.queue"
            },
            "queue": {
                "discarded": {
                    "full": 0,
                    "nf": 0
                },
                "enqueued": 0,
                "full": 0,
                "maxqsize": 0,
                "size": 0
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"input-techno1(20516)\", \"origin\": \"imtcp\", \"submitted\": 0 }",
    "sekoiaio": {
        "forwarder": {
            "input": {
                "port": "20516",
                "submitted": 0
            },
            "intake": {
                "name": "techno1"
            },
            "module": {
                "name": "input-techno1(20516)",
                "type": "imtcp"
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"input-my-techno2(*/20517/IPv4)\", \"origin\": \"imudp\", \"submitted\": 0, \"disallowed\": 0 }",
    "sekoiaio": {
        "forwarder": {
            "input": {
                "port": "*/20517/IPv4",
                "submitted": 0
            },
            "intake": {
                "name": "my-techno2"
            },
            "module": {
                "name": "input-my-techno2(*/20517/IPv4)",
                "type": "imudp"
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"ruleset-techno1\", \"origin\": \"core.queue\", \"size\": 0, \"enqueued\": 0, \"full\": 0, \"discarded.full\": 0, \"discarded.nf\": 0, \"maxqsize\": 0 }",
    "sekoiaio": {
        "forwarder": {
            "intake": {
                "name": "techno1"
            },
            "module": {
                "name": "ruleset-techno1",
                "type": "core.queue"
            },
            "queue": {
                "discarded": {
                    "full": 0,
                    "nf": 0
                },
                "enqueued": 0,
                "full": 0,
                "maxqsize": 0,
                "size": 0
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"TCP-intake.sekoia.io-10514\", \"origin\": \"omfwd\", \"bytes.sent\": 0 }",
    "sekoiaio": {
        "forwarder": {
            "intake": {
                "name": "intake.sekoia.io-10514"
            },
            "module": {
                "name": "TCP-intake.sekoia.io-10514",
                "type": "omfwd"
            },
            "output": {
                "bytes": {
                    "sent": 0
                }
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

{
    "message": "{ \"name\": \"resource-usage\", \"origin\": \"impstats\", \"utime\": 134625, \"stime\": 20397, \"maxrss\": 13888, \"minflt\": 1510, \"majflt\": 0, \"inblock\": 0, \"oublock\": 272, \"nvcsw\": 347, \"nivcsw\": 31, \"openfiles\": 15 }",
    "sekoiaio": {
        "forwarder": {
            "intake": {
                "name": "usage"
            },
            "module": {
                "name": "resource-usage",
                "type": "impstats"
            },
            "stats": {
                "openfiles": 15,
                "stime": 20397,
                "utime": 134625
            }
        }
    },
    "user": {
        "domain": "SEKOIA.IO"
    }
}

Extracted Fields

The following table lists the fields that are extracted, normalized under the ECS format, analyzed and indexed by the parser. It should be noted that infered fields are not listed.

Name	Type	Description
sekoiaio.forwarder.action.failed	number	Number of messages failed in the action
sekoiaio.forwarder.action.processed	number	Number of messages processed by the action
sekoiaio.forwarder.action.resumed	number	Number of messages resumed by the action
sekoiaio.forwarder.action.suspended	number	Number of messages suspended by the action
sekoiaio.forwarder.input.port	keyword	Input port use by the forwarder
sekoiaio.forwarder.input.submitted	number	Number of messages handled by the forwarder input
sekoiaio.forwarder.intake.name	keyword	Intake name defined in the forwarder
sekoiaio.forwarder.module.name	keyword	This identifies the module name for which the statistics are being reported
sekoiaio.forwarder.module.type	keyword	This identifies the module type for which the statistics are being reported
sekoiaio.forwarder.output.bytes.sent	number	Number of bytes sent to Sekoia
sekoiaio.forwarder.queue.discarded.full	number	Number of messages discarded because the queue was full
sekoiaio.forwarder.queue.discarded.nf	number	Number of messages discarded because the queue was nearly full
sekoiaio.forwarder.queue.enqueued	number	Total number of messages enqueued into this queue since startup
sekoiaio.forwarder.queue.full	number	Number of times the queue was actually full and could not accept additional messages
sekoiaio.forwarder.queue.maxqsize	number	The maximum amount of messages that have passed through the queue since rsyslog was started
sekoiaio.forwarder.queue.size	number	Number of messages in the queue
sekoiaio.forwarder.stats.openfiles	number	Number of file handles used by rsyslog
sekoiaio.forwarder.stats.stime	number	The amount of system CPU time consumed by rsyslog, in milliseconds.
sekoiaio.forwarder.stats.utime	number	The amount of user CPU time consumed by rsyslog, in milliseconds.
user.domain	keyword	Name of the directory the user is a member of.

For more information on the Intake Format, please find the code of the Parser, Smart Descriptions, and Supported Events here.