Files Inputs

Enter a descriptive label for this input. 

When you create a workflow, you will use this label to enter your input into the workflow. 

billing,errorcheck is the default label. 

This parameter is required. 

Review the following example: 

labels: "billing,errorcheck"

path

Enter the full path to the file (or files) that you want Edge Delta to monitor. 

• Wildcards are supported.
• If you want the agent to process lines for a specific line separation rule (not for New Line("\n")), then you need to define a "line_pattern" regex rule.

This parameter is required. 

Review the following example: 

path: "/etc/systemd/system/billingservice/*.log"

Exclude

Enter a path (in regex) to exclude matched patterns.

This parameter is optional. 

Review the following example: 

    - labels: "billing,errorcheck"
      path: "/etc/systemd/system/billingservice/*.log"
      exclude:
        - "/etc/systemd/system/billingservice/test.log"
        - "/etc/systemd/system/billingservice/dev.log"

auto_detect_line_pattern

Enter true (or false) to enable auto line detection for log messages with multiple lines. 

This parameter is optional. 

Review the following example: 

auto_detect_line_pattern: true

boost_stracktrace_detection

Enter true (or false) to fallback (troubleshoot) missing content that resulted from the Auto Line Detection parameter. 

This parameter is optional. 

Review the following example: 

boost_stacktrace_detection: true

separate_source

Enter true (or false) to have the agent treat each file that matches the glob expression as a separate source. 

By default, this option is disabled; all files that match the glob path will be treated as a single source. 

This parameter is optional. 

Review the following example: 

separate_source: true 

filters

Select (or enter) an existing filter to add to this input. 

To learn how to create a filter, see Filters.

This parameter is optional. 

Review the following example: 

filters:
  - info
  - not_trace
  - mask_card
  - mask_password

enrichments

You can use this parameter to enrich data with specified extracted fields. 

To learn how to enrich data from inputs, see Enrich Input Data.

This parameter is optional. 

Review the following example: 

   - labels: k8s_log
      path: /var/logs/anyDir/MyApp/users/MyPodID/transaction.log
      enrichments:
        from_path:
          field_mappings:
            - field_name: application
              pattern: /var/logs/anyDir/(?:(.+)/)?users/.*

add_ingestion_time

Mark (or enter) true or false to ingest a timestamp if the input format is in JSON.

This parameter is optional. 

Review the following example: 

- labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
      add_ingestion_time: true
      skip_ingestion_time_on_failure: true 

skip_ingestion_time_on_failure

Mark (or enter) true or false to skip the ingestion of the timestamp when the input is broken or in an invalid format. 

This parameter is optional. 

Review the following example: 

skip_ingestion_time_on_failure: true

late_arrival_handling

You can use this parameter to configure how to accommodate delayed data. 

Specifically, you can use the ignore_after parameter to configure when to ignore metrics or pattern logs.

For example, for ignore_after, if you enter 15m, then the input will ignore logs whose timestamp is older than 15 minutes. 

This parameter is optional. 

Review the following example:  

late_arrival_handling:
  rule_metrics:
    ignore_after: 15m
  patterns:
    ignore_after: 4h
    report_with_original_timestamp: true

late_arrival_handling

You can use this parameter to configure how to accommodate delayed data. 

Specifically, you can use the ignore_after parameter to configure when to ignore metrics or pattern logs.

For example, for ignore_after, if you enter 15m, then the input will ignore logs whose timestamp is older than 15 minutes. 

This parameter is optional. 

Review the following example:

      late_arrival_handling:
        rule_metrics:
          ignore_after: 15m
        patterns:
          ignore_after: 4h
          report_with_original_timestamp: true

source_type

Select (or enter) a source type.

Within inputs, a source type tells the Edge Delta agent which specific stream to monitor and extra logs from. 

You can select (or enter) Docker, ECS, File, K8s. or Custom. 

This parameter is optional. 

Review the following example: 

source_detection:
  source_type: "Docker"
  optional: false
  field_mappings:
    docker_container_id: "docker.id"
    docker_container_image: "docker.image"

optional

Enter true or false to ingest (or discard) logs with a failed source detection. 

Enter true to ingest logs with the original source information, despite a failed source detection.

Enter false to discard logs with a failed source detection.

This parameter is optional. 

Review the following example: 

optional: false

source_detection:

  processing_mode

There are 2 types of processing modes:

• json
• regex

If you enter json, then you must enter the JSON path as a value for each field mapping. 

If you enter regex, then you must enter a regex pattern with one capturing group named field, such as

• "path (?P<field>\w+)" 

This parameter is optional. 

Review the following example:

source_detection:
        source_type: "Custom"
        optional: false
        processing_mode: regex
        field_mappings:
          namespace: namespace (?P<field>\w+)
          serviceName: service (?P<field>\w+)
          roleName: user_role (?P<field>\w+)
          systemType: system (?P<field>\w+)

docker_mode

Enter true (or false) to collect Docker container standard output logs on a file with the JSON file logging driver.

To learn more, review this document from Docker.

This parameter is optional. 

Review the following example: 

    - labels: "docker,my_container"
      path: "/var/lib/docker/my_container/*.log"
      docker_mode: true