Overview
This input type allows you to specify a set of files for Edge Delta to monitor.
Review Sample Configuration
Review the following sample configuration:
files:
- labels: "billing,errorcheck"
path: "/billing/logfolder1/*.log"
- labels: "billing,errorcheck"
path: "/etc/systemd/system/billingservice/*.log"
exclude:
- "/etc/systemd/system/billingservice/test.log"
- "/etc/systemd/system/billingservice/dev.log"
auto_detect_line_pattern: true
boost_stacktrace_detection: true
enable_persisting_cursor: true
filters:
- info
- not_trace
- mask_card
- mask_password
- labels: "docker,my_container"
path: "/var/lib/docker/my_container/*.log"
docker_mode: true
- labels: "app,service_a"
path: "/var/log/service_a.log"
line_pattern: "^MMM dd, yyyy hh:mm:ss"
late_arrival_handling:
rule_metrics:
ignore_after: 15m
patterns:
ignore_after: 4h
report_with_original_timestamp: true
filters:
- source-detection-docker
- labels: k8s_log
path: /var/logs/anyDir/MyApp/users/MyPodID/transaction.log
filters:
- enrichment-full
- labels: "billing,errorcheck"
path: "/billing/logfolder1/*.log"
separate_source: true
- labels: "billing,errorcheck"
path: "/billing/logfolder1/*.log"
add_ingestion_time: true
skip_ingestion_time_on_failure: true
Review Parameters
Review the following parameters that you can configure in the Edge Delta App.
labels
Required
Enter a descriptive label for this input.
When you create a workflow, you will use this label to enter your input into the workflow.
billing,errorcheck is the default label.
Review the following example:
labels: "billing,errorcheck"
path
Required
Enter the full path to the file (or files) that you want Edge Delta to monitor.
- Wildcards are supported.
- If you want the agent to process lines for a specific line separation rule (not for New Line("\n")), then you need to define a "line_pattern" regex rule.
Review the following example:
path: "/etc/systemd/system/billingservice/*.log"
exclude
Optional
Enter a path (in regex) to exclude matched patterns.
Review the following example:
- labels: "billing,errorcheck" path: "/etc/systemd/system/billingservice/*.log" exclude: - "/etc/systemd/system/billingservice/test.log" - "/etc/systemd/system/billingservice/dev.log"
auto_detect_line_pattern
Optional
Enter true (or false) to enable auto line detection for log messages with multiple lines.
Review the following example:
auto_detect_line_pattern: true
boost_stracktrace_detection
Optional
Enter true (or false) to fallback (troubleshoot) missing content that resulted from the Auto Line Detection parameter.
Review the following example:
boost_stacktrace_detection: true
separate_source
Optional
Enter true (or false) to have the agent treat each file that matches the glob expression as a separate source.
By default, this option is disabled; all files that match the glob path will be treated as a single source.
Review the following example:
separate_source: true
filters
Optional
Enter an existing filter to add to this input.
To learn how to create a filter, see Filters.
Review the following example:
filters: - info - not_trace - mask_card - mask_password
add_ingestion_time
Optional
Enter true (or false) to ingest a timestamp if the input format is in JSON.
Review the following example:
add_ingestion_time: true
skip_ingestion_time_on_failure
Optional
Enter true or false to skip the ingestion of the timestamp when the input is broken or in an invalid format.
Review the following example:
skip_ingestion_time_on_failure: true
late_arrival_handling
Optional
You can use this parameter to configure how to accommodate delayed data.
Specifically, you can use the ignore_after parameter to configure when to ignore metrics or pattern logs.
For example, for ignore_after, if you enter 15m, then the input will ignore logs whose timestamp is older than 15 minutes.
Review the following example:
late_arrival_handling: rule_metrics: ignore_after: 15m patterns: ignore_after: 4h report_with_original_timestamp: true
docker_mode
Optional
Enter true (or false) to collect Docker container standard output logs on a file with the JSON file logging driver.
To learn more, review this document from Docker.
Review the following example:
docker_mode: true