Docs › Managing devices › Gateway Logs

Gateway Logs

A timestamped series of log messages can provide insight into device operation. Time-series event logging is important for security and drive monitoring. The log processing feature in the Izuma Edge gateway stack supports log collection of various standard Linux log sources.

Log processing

FluentBit is a log processor, stream processor and forwarder for Linux and various other operating systems. Izuma Edge uses FluentBit to collect logs from various input plugins for its services, parse them and forward them to Izuma Device Management.

Configuration

The FluentBit daemon runs on the gateway as a system service managed by systemd. The service parameters are defined in the systemd unit file or config file for the FluentBit service that contains the FluentBit log processing chain. The FluentBit daemon is configured through this standard configuration file. The config file holds information on the input plugins used to collect logs, how to parse them and where to forward the collected log stack.

This is an example for capturing CPU usage logs in Linux and uses an HTTP output plugin to forward data to Izuma Device Management. It uses the record modifier filter to append important fields to log data, such as application name, log level and time stamp:

[SERVICE]
    Flush     5
    Daemon    off
    Log_Level debug

@SET cpu_tag=cpu

[INPUT]
    Name cpu
    Tag  ${cpu_tag}
[FILTER]
    Name record_modifier
    Match *
    Record type ${cpu_tag}
    Record app_name fluentbit
    Record level INFO

[FILTER]
    Name nest
    Match *
    Operation nest
    Wildcard user_p*
    Wildcard system_p*
    Wildcard cpu*
    Nest_under message_json

[OUTPUT]
    Name  http
    Match *
    Host  gateways.local
    Port  8080
    URI   /v3/device-logs
    Format json
    json_date_format iso8601
    json_date_key date_time
    Retry_Limit     2

Note: The host gateways.local will be resolved as 127.0.0.1, which Izuma Edge set up to forward traffic to the gateways cloud domain.

By default on Izuma Edge, FluentBit is configured with:

Input endpoints: CPU; MEM; and systemd services, such as Edge Core, edge-proxy service, pelion-relay-term, Maestro, kubelet, Docker and wait-for-pelion-identity.
Output endpoint: HTTP to publish logs to Izuma hosted logging service.

Dynamic log settings or remote configuration

You can define your own processing chains to customize the data collected from a device to meet your particular needs. You can configure the processing chain at run time through a configuration file.

The Izuma Edge gateway advertises the features supported by the platform using Pelion's reserved FeatureMgmt LwM2M object 33457 with three resources:

0: featureID.
1: enabled.
2: config.

This provides an interface for discovering the gateway's feature capabilities and to manage the configuration of enabled features. Refer to the Maestro configuration documentation to learn more about the gateway capability discovery feature. The FluentBit logging feature is also one such Feature Management instance. FeatureMgmt config resource for the FluentBit service allows you to remotely view the current logging configuration and also push a config update using the Izuma Device Management service APIs. When such an update is triggered, the edge gateway can instantly apply the changed configuration.

Log service

Gateway Logs Service stores logs in one location, so you can discover and resolve issues when debugging. The gateway sends its logs to the service, which stores and filters them. You can then review the logs and sort them based on the filters Gateway Logs Service provides.

The supported filters are:

Account ID.
Device ID.
Log ID.
Application name.
Log type.
Log time stamp.
Log severity level, such as debug, trace, info, warning, error and critical.

Prerequisites

To use Gateway Logs Service, you need an access key to access Izuma Edge gateway services.

Using the service

With Gateway Logs Service, you can:

Security considerations

Gateway Logs Service only accepts HTTPS connections, which are secured by Secure Sockets Layer (SSL) protocol. It provides privacy and data integrity between two or more communicating applications.
You can only view gateway logs if you are part of the account tied to that gateway.
For gateway domain requests, the gateway authenticaiton server has to confirm whether the request is from a valid gateway.
All requests made to the API domain must pass through the API gateway, which ensures the request contains a valid access token. The Gateway Logs Service assumes the API gateway handles authentication for API domain requests.

Documentation