Release Notes

George Alpizar
George Alpizar
  • Updated

June 6, 2022 - Agent Version 0.1.25

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Updates Agent Installation

The logging directory for the Edge Delta agent has been updated.

Specifically, if you install the agent via the installation script, then:

  • For Windows agents, logs can be found under %AppData%/edgedelta.

  • For all other operating systems, logs can be found under /var/logs/edgedelta.


Updated Input Enrichment

In the Edge Delta App, input enrichment has been updated. Specifically, the field_name parameter under dynamic enrichment can be applied to a template.

Review the following example:

      enrichments:
        dynamic:
          field_mappings:
            - field_name: `{{if eq .controllerKind "replicaSet"}}kube_deployment{{else}}kube_{{.controllerKind}}{{end}}`
              value: "{{.controllerName}}"

Note

For more complicated templates that include if / else statements or range statements, you must use bracket as a delimiter.


Updated source_detection Parameter

The source_detection parameter for inputs has been updated.

Specifically, source_detection now supports custom as a source_type. When you enter custom, you must configure the field_mappings parameter with a key-value pair. 

Review the following example:

    - labels: "my-kafka-events"
      endpoint: "something"
      topic: "topic"
      group_id: "my-group"
      sasl:
        username: kafka_username
        password: p@ssword123
        mechanism: PLAIN  
      source_detection:
        source_type: "Custom"
        optional: false
        field_mappings:
          namespace: "kubernetes.namespace"
          serviceName: "service"
          roleName: "user.role"
          systemType: "system"

Additionally, source_detection now supports regex as a processing_mode.

Review the following example:

    - labels: "my-kafka-events"
      endpoint: "something"
      topic: "topic"
      group_id: "my-group"
      sasl:
        username: kafka_username
        password: p@ssword123
        mechanism: PLAIN 
      source_detection:
        source_type: "Custom"
        optional: false
        processing_mode: regex
        field_mappings:
          namespace: namespace (?P<field>\w+)
          serviceName: service (?P<field>\w+)
          roleName: user_role (?P<field>\w+)
          systemType: system (?P<field>\w+)

May 10, 2022 - Agent Version 0.1.24

This agent release contains backend enhancements and bug fixes.


May 6, 2022 - Agent Version 0.1.23

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Updated Splunk Streaming Output

In the Edge Delta App, the Splunk streaming output has been updated to support custom tags via the custom_tags parameter.

You can use this parameter to define key-value pairs that are streamed with every request.

Review the following sample configuration:

- name: my-splunk
      type: splunk
      endpoint: "://:/"
      token: "32-character GUID token"
      custom_tags:
        "app": "test"
        "region": "us-west-2"
        "File Path": "{{.FileGlobPath}}"
        "K8s PodName": "{{.K8sPodName}}"
        "K8s Namespace": "{{.K8sNamespace}}"
        "K8s ControllerKind": "{{.K8sControllerKind}}"
        "K8s ContainerName": "{{.K8sContainerName}}"
        "K8s ContainerImage": "{{.K8sContainerImage}}"
        "K8s ControllerLogicalName": "{{.K8sControllerLogicalName}}"
        "ECSCluster": "{{.ECSCluster}}"
        "ECSContainerName": "{{.ECSContainerName}}"
        "ECSTaskVersion": "{{.ECSTaskVersion}}"
        "ECSTaskFamily": "{{.ECSTaskFamily}}"
        "DockerContainerName": "{{.DockerContainerName}}"
        "ConfigID": "{{.ConfigID}}"
        "Host": "{{.Host}}"
        "Source": "{{.Source}}"
        "SourceType": "{{.SourceType}}"
        "Tag": "{{.Tag}}"

To learn more, see Review Parameters for Stream Outputs and Integrations .


Updated Agent Settings

In the Edge Delta App, the Agent Settings section has been updated with new parameters.

Review the following parameters:

Visual Editor YAML Description Default Value Required or Optional 
Not applicable max_file_per_glob_path

Enter the maximum number of files to tail, per glob path.

100 Optional
Not applicable forget_file_after

Enter a length of time to drop files that have not been modified.

1h Optional
Not applicable total_seek_capacity

Enter the maximum size that tailers can seek concurrently.

5MB Optional
Not applicable max_seek_size

Enter the maximum size that a tailer can seek, per second.

  Optional
Not applicable source_discovery_interval

Enter how often the source discovery is invoked.

5s Optional
Not applicable file_tailer_buffer_size:

Enter the maximum number of logs that a file tailer can store in its memory until the logs are ingested by the agent’s internal router.

If the router is busy and cannot ingest the logs and the tailer’s buffer is reached, then the seeking will be blocked.

1000 Optional
Not applicable router_per_source_buffer_size

Enter the maximum number of logs that an agent’s internal router can store in its memory, per source.

1000 Optional
Not applicable archive_flush_interval

Enter a time frame to flush and send logs to a configured archiving destination.

 

5m Optional
Not applicable archive_max_byte_limit

Enter the maximum number of bytes that can be buffered (in memory) before a flush is triggered to an archive destination.

26MB Optional

Review the following sample configuration:

  max_file_per_glob_path: 100
  forget_file_after: 1h
  total_seek_capacity: "5 MB"
  max_seek_size: "4 MB"
  source_discovery_interval: 5s
  file_tailer_buffer_size: 1000
  router_per_source_buffer_size: 1000
  archive_flush_interval: 5m
  archive_max_byte_limit: "16MB"

To learn more, see Agent Settings.


May 2, 2022 - Agent Version 0.1.22

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Updated Enrichment Options

In the Edge Delta App, data enrichment options have been updated to address failed or failing sources.

To troubleshoot potential mapping failures, you can configure the failure_behavior parameter. 

Review the following options:

failure_behavior option Description
stop_enrichment This option will terminate the enrichment.
drop_source This option will terminate the enrichment and also stop tailing the source.
skip_failing_fields

This option will skip over the detected failed mapping in the enrichment.

This option is the default value.

Additionally, you can use the fallback_value parameter to troubleshoot. Specifically, if mapping fails based on the value or json_path parameter, then the configured value for fallback_value will be used until the agent confirms that the mapping has failed. 

Review the following example: 

      enrichments:
        failure_behavior: stop_enrichment
        dynamic:
          field_mappings:
            - field_name: "service"
              value: '{{".labels.service"}}'
            - field_name: "source"
              value: '.annotations.kubernetes.io/{{.container_name}}.logs'
              json_path: "[0].source"
              fallback_value: '{{".short_container_image"}}'

To learn more, see Enrich Input Data.


Updated Source Types

In the Edge Delta App, streaming outputs have been updated.

Specifically, for the source_type parameter, you can now enter custom.

Previously, this parameter only supported K8s, Docker, ECS, and File.

When you enter custom, you must add field_mappings parameters to indicate the file source.

Review the following example:

- labels: "my-kafka-events"
      endpoint: "something"
      topic: "topic"
      group_id: "my-group"
      sasl:
        username: kafka_username
        password: p@ssword123
        mechanism: PLAIN 
      source_detection:
        source_type: "Custom"
        optional: false
        field_mappings:
          namespace: "kubernetes.namespace"
          serviceName: "service"
          roleName: "user.role"
          systemType: "system"

To learn more, see Review Parameters for Stream Outputs and Integrations .


Updated File Inputs

In the Edge Delta App, the file input type has been updated.

Specifically, you can use the newly created exclude parameter to enter a glob path to exclude matched patterns.

Review the following example:

files:
    - labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
    - labels: "billing,errorcheck"
      path: "/etc/systemd/system/billingservice/*.log"
      exclude:
        - "/etc/systemd/system/billingservice/test.log"
        - "/etc/systemd/system/billingservice/dev.log"

April 27, 2022 - Agent Version 0.1.21

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Updated Splunk Output

In the Edge Delta App, the Splunk streaming output has been updated with the ability to send data in a JSON format.

Specifically, to use this option, you must update the endpoint parameter to point to Splunk's API services/collector/raw, instead of services/collector/event.

Review the following sample configuration:

    - name: splunk-integration
      type: splunk
      endpoint: ..../services/collector/raw
      token: ....
      features: log,metric,edac,cluster,alert
      index: rehydration

New Filter Type

In the Edge Delta App, you can use the newly created JSON Field Extractor filter to extract a field's value and replace the whole JSON content with the field's value.

Extract_JSON_Field_Filter.png

Review the following sample configuration: 

  - name: extract_severity
    type: extract-json-field
    field_path: "severity"
  - name: extract_first_data
    type: extract-json-field
    field_path: "records.[0].data"

To learn more, see JSON Field Extractor Filters.


Updated Enrichments for AWS ECS Inputs

The configurations to enrich input data have been updated.

Specifically, you can use the dynamic parameter to enrich input data from AWS ECS.

In the agent configuration, you can

To obtain data from an AWS EC2 instance, in the value parameter, you must enter aws-instance

Review the following sample configuration:

      enrichments:
        dynamic:
          field_mappings:
            - field_name: "instance_id"
              value: '{{".aws-instance.instance-id"}}'
            - field_name: "instance_type"
              value: '{{".aws-instance.instance-type"}}'
            - field_name: "cluster_name"
              value: '{{".aws-instance.cluster-name"}}'
            - field_name: "ec2launchtemplate_id"
              value: '{{".aws-instance.ec2launchtemplate-id"}}'
            - field_name: "ec2launchtemplate_version"
              value: '{{".aws-instance.ec2launchtemplate-version"}}'
            - field_name: "inspector_enabled"
              value: '{{".aws-instance.inspector-enabled"}}'
            - field_name: "cluster_autoscaler_enabled"
              value: '{{".aws-instance.cluster-autoscaler-enabled"}}'
            - field_name: "autoscaling_groupName"
              value: '{{".aws-instance.autoscaling-groupName"}}'
            - field_name: "nodegroup_name"
              value: '{{".aws-instance.nodegroup-name"}}'
            - field_name: "ec2_fleet_id"
              value: '{{".aws-instance.ec2-fleet-id"}}'

To learn how to enrich input, see Enrich Input Data.

To learn how to retrieve instance metadata, review this document from AWS.


New Filter Type

In the Edge Delta App, you can use the newly created Split with Delimiter filter to match, then split a single log into multiple logs.

For example, the abc\n\ndef\nxyz\n log would split into 3 separate logs (abc, def, xyz), based on the configured delimiter, (newline character ( \n )).

Review the following sample configuration:

 - name: split_logs_using_specified_delimiter
    type: split-with-delimiter
    delimiter: ","

To learn more, see Split Lines Filters.


Updated Archiving Outputs

In the Edge Delta App, archiving outputs have been updated.

Specifically, you can use the new use_native_compression option to compress data, but not metadata.

Use_Native_Compression.png

This option can be useful with big data cloud applications, such as AWS Athena and Google BigQuery.

Note

To use this parameter, you must set the encoding parameter to parquet

Review the following sample configuration:

    - name: my-minio
      type: minio
      access_key: my_access_key_123
      secret_key: my_secret_key_123
      endpoint: play.minio.com:9000
      bucket: ed-test-bucket-minio
      disable_ssl: true
      s3_force_path_style: true
      encoding: parquet 
      compression: zstd 
      use_native_compression: true

To learn more, see Archive Outputs and Integrations Overview.


Updated Cluster Processors

In the Edge Delta App, cluster processors have been updated with a new configuration.

Specifically, you can use the newly created include_pattern_info_in_samples parameter to include pattern information in a cluster sample, such as patterns, pattern counts, and sentiment scores.

Include_Pattern_Info_In_Cluster_Sample.png

Review the following sample configuration:

processors:
  cluster:
    name: clustering
    num_of_clusters: 100
    samples_per_cluster: 20
    reporting_frequency: 30s
    retention: 10m
    cpu_friendly: true
    throttle_limit_per_sec: 200
    include_pattern_info_in_samples: true 

To learn more, see Cluster Processors.


New Input type

In the Edge Delta App, NATS JetStream is now a supported input type.

This input type allows you to specify a NATS stream subscription for Edge Delta to monitor. 

NATs_JetStream_Input.png

Review the following sample configuration: 

  nats:
    - labels: "my-nats-normal"
      input_mode: "normal"
      consumer_mode: "pull"
      cluster_url: "nats://localhost:4222"
      stream_name: "example-stream"
      subject: "example-subject-1"
      timeout: 1m
      ack_wait_duration: 10s
    - labels: "my-nats-distributed"
      input_mode: "distributed"
      consumer_mode: "push"
      cluster_url: "nats://localhost:4222"
      stream_name: "example-stream"
      subject_prefix: "example-subject"
      total_agent_count: 5
      total_subject_count: 10
      should_split_lines: true
      timeout: 1m
      disable_acks: true

To learn more, see NATS JetStream Inputs.


Updated Datadog Streaming Output

In the Edge Delta App, the Datadog Streaming Output has been updated with buffered-related options.

Specifically, you can use the following, newly created parameters to configure the output’s buffering behavior:

Parameter  Description
buffer_ttl

Enter a length of time to retry failed streaming data.

After this length of time is reached, the failed streaming data will no longer be tried.

This parameter is optional. 

Review the following example: 

buffer_ttl: 2h
buffer_path

Enter a folder path to temporarily store failed streaming data.

The failed streaming data will be retried until the data reaches its destinations or until the Buffer TTL value is reached.

If you enter a path that does not exist, then the agent will create directories, as needed.

This parameter is optional.

Review the following example:

buffer_path: /var/log/edgedelta/pushbuffer/
buffer_max_bytesize

Enter the maximum size of failed streaming data that you want to retry.

If the failed streaming data is larger than this size, then the failed streaming data will not be retried.

This parameter is optional.

Review the following example:

buffer_max_bytesize: 100MB

To learn more, see Review Parameters for Stream Outputs and Integrations .


April 12, 2022 - Agent Version 0.1.20

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Updated Log Enrichment

In the Edge Delta App, log enrichment features have been updated to now support enrichment from Kubernetes annotations. 

Specifically, you can use the from_k8s parameter to enrich streaming data with K8s attributes.

You can enter a podnamespace, or node attributes.

Review the following example: 

        from_k8s:
          pod_identifier_pattern: /var/logs/anyDir/MyApp/users/(?:(.+)/)/.*
          field_mappings:
            - field_name: instance_id
              pod_attribute: pod
              transformers:
                # replace all "source" matches with "target"
                - source: "-"
                  target: "_"
                  type: "replace"
                # remove all "test" words
                - source: "test*"
                  target: ""
                  type: "regex"
            - field_name: namespace
              pod_attribute: namespace
            # fields from labels should have pod_attribute start with "labels."
            - field_name: service
              pod_attribute: labels.service

To learn more, review the Enrich Input Data section in the Inputs document.


Updated Numeric Capture (Regexes) Processors

In the Edge Delta App, the Numeric Capture (Regexes) processor has been updated to support multiplication and division for numeric value captures.

Specifically, you can use the newly created value_adjustment_rules parameter to create a rule per capture group. 

The rule must follow the "(*|/)" format where:

  • An asterisk ( * ) represents multiplication
  • A slash ( / ) represents division

Review the following example: 

    - name: "flog"
      pattern: " (?P\\d+) (?P\\d+)$"
      value_adjustment_rules:
        responsesize:
          operator: "/"
          operand: 1000.0

To learn more, review the Numeric Capture (Regexes) Processor section in the Processors document.


Updated Edge Delta Agent

In an effort to improve security during agent installation, makeself --sha256 option has been enabled on agent deployments.

Specifically, makeself performs md5 and crc checks for content integrity.

Starting with version 0.1.20 of the agent, this upgraded security measure update will be included in all agent deployments.


Updated Sumo Logic Output

In the Edge Delta App, the Sumo Logic streaming output has been updated. 

Specifically, you can use the newly created send_as_json parameter to send data in a JSON format, which allows the fields to be auto-parsed and extracted in Sumo.

Review the following example: 

    - name: sumo-us-2
      type: sumologic
      endpoint: '{{ Env "EMPTY" "https://endpoint4.collection.us2.sumologic.com/receiver/v1/http/XYZ" }}'
      send_as_json: true

To learn more, see Review Parameters for Stream Outputs and Integrations.


Updated Enriched Data for AWS

In the Edge Delta App, you can use the dynamic enrichment feature to obtain data from an AWS EC2 instance.

Specifically, in the value parameter, you must enter aws-instance

Review the following example:

      enrichments:
        dynamic:
          field_mappings:
            # if the field value starts with "aws-instance" then instance metadata is get from aws ec2 instance.
            # for more info ref: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
            - field_name: "instance_id"
              value: '{{".aws-instance.instance-id"}}'
            - field_name: "instance_type"
              value: '{{".aws-instance.instance-type"}}'

To learn more, review the Enrich Input Data section of the Inputs document. 

For additional information, please review the Retrieve instance metadata document from Amazon. 


Updated Enriched Data for JSON

In the Edge Delta App, the from_logs enrichment feature has been with the json_path parameter. 

You can use the json_path parameter to enrich data with fields extracted from JSON logs.

Review the following example:

        # from_logs is used to enrich data with fields extracted from logs
        from_logs:
          field_mappings:
            - field_name: component
              # extracting using json_path is also supported
              json_path: fields.[1].component

To learn more, review the Enrich Input Data section of the Inputs document. 


Updated Workflows

In the Edge Delta App, you can use the newly created enabled_hosts parameter to limit the workflow to specific hosts.

With this parameter, the workflow will only run for specified agent hosts. 

Review the following example:

  enabled_hosts_workflow:
    description: "runs only specified hosts"
    input_labels:
      - system
      - docker
      - agent
      - infa-processes
    filters:
      - info
    destinations:
      - '{{ Env "TEST_SUMO" }}'
    enabled_hosts:
      - my.host.us1
      - my.host.us2

To learn more, see Workflows.


New Filter / Process Type

In the Edge Delta App, you can use the newly created OTLP filter / processor to process OTLP (Open Telemetry) logs. 

Review the following example:

  - name: opentelemetry_trace_filter
    type: buffered-otlp-trace-processor
    trace_deadline: 1m
    should_filter_traces: true
    failure_path: "attributes.result_code"
    failure_value_pattern: "(4|5)xx"
    latency_threshold: 2500.0
    success_sample_rate: 0.1

To learn more, see Filters.


March 21, 2022 - Agent Version 0.1.19

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


New Streaming Destination - GCP Cloud Monitoring

In the Edge Delta App, GCP Cloud Monitoring is now a supported streaming output.

The GCP Cloud Monitoring output will stream custom Google Cloud metrics to a Cloud project.

In the app, you can use the visual editor or YAML file to add GCP Cloud Monitoring to an agent configuration.

Cloud_Monitoring_Streaming_Output.png

Note

GCP Cloud Monitoring was previously known as GCP Stackdriver.

To learn more, see Review Parameters for Stream Outputs and Integrations.


New Flush Mode

In the Edge Delta App, you can use the newly created custom_local_per_group flush mode to specify custom groups that should flush together if one of the groups triggers an alert.

In other words, if you set up multiple inputs, and one input triggers an alert, then all (or selected) inputs will flush.

New_Flush_Mode.png

Review the following example:

agent_settings:
  tag: sett_test_custom
  log:
    level: debug
  capture_flush_mode: custom_local_per_group
  capture_flush_custom:
    label_grouping:
      group1:
        - file1
        - file2
      group2:
        - file1
        - file3

inputs:
  files:
    - labels: "file1"
      path: "test1.log"
    - labels: "file2"
      path: "test2.log"
    - labels: "file3"
      path: "test3.log"
    - labels: "file4"
      path: "test4.log"

outputs:
  streams:
    - name: sumo
      type: sumologic
      endpoint: "https://endpoint4.collection.us2.sumologic.com/receiver/v1/http/ZaVnC4dhaV1ozOeONNQ8LuYTYUj7SaKgr6dt1ueSTOc6mMS2pQz9BM169sb8_UQs5IRaqaRcbpKdI4Tms9S5La9ZFRTL_bf-Ptf_I5ICXcQz2WEQg0fNfA=="
      features: alert

processors:
  regexes:
    - name: "error-regex" 
      pattern: "error|ERROR|problem|ERR|Err|POST|hostname|GET"
      interval: 10s
      retention: 1h
      trigger_thresholds:
        upper_limit_per_interval: 3

workflows:
  error-anomaly-workflow:
    input_labels:
      - file1
      - file2
      - file3
      - file4
    processors:
      - error-regex
    destinations:
      - sumo

Based on the above example, the following actions will take place: 

  • If an alert is triggered for test1.log, then the file1, file2, and file3 sources will be flushed together. 
  • If an alert is triggered for test2.log, then the file1 and file2 sources will be flushed together. 
  • If an alert is triggered for test3.log, then the file1 and file3 sources will be flushed together. 
  • If an alert is triggered for test4.log, then the file4 source will be flushed. 
    • Since file4 was not specified in any group and a fallback_mode was not provided, the agent will use the default fallback_mode local_per_source and only flush file4.
  • When fallback_mode: local_all is added, and alert is triggered for test4.log, then all sources will be flushed. 

To learn more about filters, see Agent Settings.


New Filter / Processor Type 

In the Edge Delta App, you can use the newly created buffered-elastic-apm filter / processor to process Elastic APM logs. 

Review the following example:

  - name: elastic_apm_trace_filter
    type: buffered-elastic-apm-processor

To learn more about processors, see Processors.


New Filter Type - base64 decoder

In the Edge Delta App, you can use the newly created base64 decoder filter type to decode base64 encoding.

This update helps to support a base64 encoded input on Edge Delta’s hosted collector. In other words, you can attach this filter to a source to display logs that are base64 encoded.

Review the following example:

  - name: base64_decoder
    type: base64-decode

To learn more about filters, see Filters.


New Monitor Types 

You can use the new Pattern Alert and Skyline Alert monitors to trigger an alert for negative patterns. 

If an alert is triggered, then the monitor will create a finding. 

Pattern_and_Skyline_Alert_Monitors.png

Pattern_Alert_Monitor.png

To learn more, see Patterns.


New feature - Suppress Notifications

In the Edge Delta App, you can use the newly created Finding Status option to suppress notifications for a specific finding.

When you suppress a finding, the finding will no longer be displayed in the Insights page. Additionally, any future detection of the finding will not be displayed.

Suppress_Notifications.png

To learn more, see Patterns.


New Filter Type - APM

 In the Edge Delta App, you can use the newly created APM filter to process Elastic APM logs.

This filter type samples failed and high-latency traces with successful traces and a sampling probability. 

Review the following sample configuration:

  - name: elastic_apm_trace_filter
    type: buffered-elastic-apm-processor
    payload_separator: "-----------------"
    enabled_types: "transaction,span,error,metricset"
    optimize_types: "transaction,span"
    trace_deadline: 1m
    should_filter_traces: true
    failure_path: "transaction.result"
    failure_value_pattern: "HTTP (4|5)xx"
    transaction_latency_path: "transaction.duration"
    span_latency_path: "span.duration"
    latency_threshold: 35.5
    success_sample_rate: 0.2

To learn more, see Filters.


March 24, 2022 - New Organizations Feature

In the Edge Delta App, you can now create and join different organizations for you and your users.

At a high level, organizations can be considered as different environments, platforms, or sub-accounts that live within your main Edge Delta account.

You can create and belong to multiple organizations, as well as invite users to join various organizations.

Create_an_Organization.png

Additionally, you can switch between organizations without the need to log off and log back into the app. 

Switch_Organizations.png

To learn more, see Invite Users, Manage Permissions, Access Organizations.


March 21, 2022 - Agent Version 0.1.18

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


New Streaming Destination - ObserveInc

In the Edge Delta App, ObserveInc is now a supported streaming output.

The ObserveInc output will stream analytics and insights to your ObserveInc endpoint.

In the app, you can use the visual editor or YAML file to add ObserveInc to an agent configuration.

ObserveInc.png

To learn more, see Review Parameters for Stream Outputs and Integrations.


New Filter - Custom Attributes

In the Edge Delta App, you can use the new Custom Attributes filter to filter for custom attributes. 

Specifically, you can use the Attribute Key and Attribute Value parameters to filter for custom attributes.

Custom_Attributes_filter.png

Review the following example:

  # Custom attribute filter do all the log filtering with given key-value of the attribute
  - name: custom_attributes_filter
    type: custom-attributes
    key: service
    value: billing
  - name: negate_custom_attributes_filter
    type: custom-attributes
    key: component
    # A comma separated values to match. If any of them matches the given attribute's value then the log will be pass through
    value: credithandler,debithandler
    # Negate is also supported for attribute filter
    negate: true
  # Filtering custom attributes also support regex matching
  - name: regex_custom_attributes_filter
    type: custom-attributes
    key: level
    pattern: "error|ERROR|problem|ERR|Err"

To learn more, see Filters.


Updated Agent Settings - Log

In the Edge Delta App, the Log parameter in the Agent Settings has been updated.

Specifically, you can use the Secure Logging option to hide sensitive data from the specified agent logs, such as API keys, secrets, and authentication information. 

Secure_Logging.png

To learn more, see Agent Settings.


Updated Integration - Loki

In the Edge Delta App, the Loki integration has been updated with a new option.

Specifically, the Send Alert As Loki Log option allows you to send alerts as a log to a Loki endpoint.

Loki_updates.png

To learn more about Loki, see Review Parameters for Stream Outputs and Integrations.


New Input Type - Google Pub/Sub

In the Edge Delta App, Pub/Sub is now a supported input type.

This input type allows you to specify a Pub/Sub project for Edge Delta to monitor. Specifically, EdgeDelta will consume messages from Pub/Sub subscriptions.

In the app, you can use the visual editor or YAML file to add PubSub to an agent configuration.

PubSub.png

To learn more, see Inputs.


Updated Input - File

In the Edge Delta App, the File input type has been updated.

Specifically, there are 2 new parameters: 

  • Add Ingestion Timestamp
    • You can use this parameter to ingest a timestamp if the input format is in JSON.
  • Skip Ingestion Timestamp On Failure 
    • You can use this parameter to skip the ingestion of the timestamp when the input is broken or in an invalid format.

Updated_File_Input_Type.png

Review the following example:

files:
    - labels: "billing,errorcheck"
      path: "/billing/logfolder1/*.log"
      # ingest timestamp if input is JSON format.
      add_ingestion_time: true
      skip_ingestion_time_on_failure: true # skip ingestion time when the input is broken or invalid format.

To learn more, see Inputs.


New Filter - Combinations 

In the Edge Delta App, you can use the newly created combination filter to combine with other, existing filters. Specifically, you can use and or or terms to combine filters to create a more customized filter. 

New_Combination_Filter_Type.png

Review the following example: 

filters:
  - name: combine_two_filters
    type: combination
    operator: or
    filters_list: 
      - pattern: "INFO"
      - filter_name: error

To learn more, see Filters.


New Filter Type - Drop Json Fields

In the Edge Delta App, you can use the newly created drop-json-fields filter to filter and drop specified JSON fields. 

Drop_Json_Fields_Filter.png

Review the following example: 

filters:
  - name: drop_some_fields
    type: drop-json-fields
    field_paths: # Each field path is a dot separated path of the field (i.e. "log.source")
      - "level"
      - "details"
      - "log.source"

 


New Input Type - EDPort

In the Edge Delta App, you can use the newly created EDPort Collector Inputs input type to specify a set of ports and protocols for the agent to listen on for incoming traffic.

ED_Ports_Input.png

Review the following example: 

inputs:
  ed_ports:
    - labels: request
      port: 9000
      protocol: tcp
      read_size: 1
      read_timeout: 30s
      source_detection:
        source_type: "K8s"
        optional: false
        field_mappings:
          k8s_namespace: "kubernetes.namespace"
          k8s_pod_name: "kubernetes.pod.name"
          k8s_container_name: "kubernetes.container.name"
          k8s_container_image: "kubernetes.container.image"
      enrichments:
        from_logs:
          field_mappings:
            - field_name: environment
              json_path: kubernetes.tags.env

To learn more, see Inputs.


February 11, 2022 - Updated Edge Delta App Design

In order to provide a better user experience, the overall look and feel of the Edge Delta App have been updated. 

While functionality within the app was not updated in this release, the app's style and design have been updated. Specifically: 

  • By default, the left-side navigation menu is collapsed
  • Font styles were updated for better readability
  • Buttons throughout the app have been increased in size to improve user accessibility 
  • Contrast between background colors and font colors have been updated for easier reading

Note

While every page has been updated with the new design, all functionality remains unchanged. 


February 1, 2022 - Agent Version 0.1.16 

While this agent release contains many enhancements and bug fixes, review the following noteworthy updates:


Multi-Threshold Support

In the Edge Delta App, you can add multi-threshold settings to an agent configuration file.

For example, you can set a threshold to generate an alert when the following conditions are met: 

  • anomaly score is > 90
  • response_time_ms.avg is > 250

To set this configuration, use the type parameter, specifically set to AND.

To better understand, review the following sample YAML:

  - name: cluster-errors-multi-threshold
    type: and
    interval: 1m
    conditions: 
    - metric_name: http_request_method_updateconfig_latency.avg
      operator: ">="
      value: 100
    - metric_name: http_request_method_deleteconfig_latency.max
      operator: ">"
      value: 125
      consecutive: 5  

To learn more, see Thresholds.


Graylog Integration

In the Edge Delta App, Graylog is now a supported streaming output.

The Graylog output will stream analytics and insights to your Graylog endpoint. In the app, you can use a YAML file to add Graylog to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


Dynatrace Integration 

In the Edge Delta App, Dynatrace is now a supported streaming output.

The Dynatrace output will stream analytics and insights to a Dynatrace environment. In the app, you can use the visual editor or YAML file to add Dynatrace to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


Updates to Datadog Integration

The Datadog Integration has been updated with a new parameter called Send Alert As Datadog Log. With this update, you can now send alerts as logs.

Datadog_Updates.png

To learn more about the parameters for a Datadog integration, see Review Parameters for Stream Outputs and Integrations.


Additional Information

This agent release contains additional enhancements and bug fixes.

To see the complete list of changes, please visit the Changelog - Agent Releases page. 


January 31, 2022 - Notification for Throttled Data

In the Edge Delta App, a notification has been created to let users know when data ingestion is being throttled.

Specifically, when a data ingestion limit is reached, the Edge Delta agent is throttled, and as a result, a notification will display at the top of the app.

New Throttle Notification.png

To learn more about these limits, please contact Edge Delta Support.


December 15, 2021 - VictorOps Integration 

In the Edge Delta App, VictorOps is now a supported triggering output.

The VictorOps output streams notifications and alerts to a VictorOps endpoint. In the app, you can use the visual editor or YAML file to add VictorOps to a configuration.

Note:

VictorOps is also known as Splunk On-Call; however, the app will refer to this output as simply VictorOps.

To learn more, see Review Parameters for Trigger Outputs and Integrations.


December 30, 2021 - AppDynamics Integration

In the Edge Delta App, AppDynamics is now a supported streaming output.

The AppDynamics output will stream analytics and insights to an AppDynamics environment. In the app, you can use the visual editor or YAML file to add AppDynamics to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 17, 2021 - OpenMetrics Integration

In the Edge Delta App, OpenMetrics is now a supported streaming output.

The OpenMetrics output will stream analytics and insights to an OpenMetrics endpoint. In the app, you can use a YAML file or visual editor to add OpenMetrics to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 16, 2021 - S3 Integration

In the Edge Delta App, S3 is now a supported streaming output.

The S3 output will stream analytics and insights to an S3 bucket. In the app, you can use a YAML file or visual editor to add S3 to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 15, 2021 - Cribl Integration 

In the Edge Delta App, Cribl is now a supported streaming output.

The Cribl output streams analytics and insights to a Cribl endpoint. In the app, you can use the visual editor or YAML file to add Cribl to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 4, 2021 - Moogsoft Integration

In the Edge Delta App, Moogsoft is now a supported triggering output.

The Moogsoft output will stream notifications and alerts to a specified Moogsoft URL. In the app, you can use the visual editor or YAML file to add Moogsoft to a configuration.

To learn more, see Review Parameters for Trigger Outputs and Integrations


December 1, 2021 - Honeycomb Integration

In the Edge Delta App, Honeycomb is now a supported streaming output.

The Honeycomb output will stream analytics and insights to a Honeycomb environment. In the app, you can use the visual editor or YAML file to add Honeycomb to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 1, 2021 - FluentD Integration

In the Edge Delta App, FluentD is now a supported streaming output.

The FluentD output will stream analytics and insights to your FluentD endpoint. In the app, you can use the visual editor or YAML file to add FluentD to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 1, 2021 - AWS CloudWatch Event Logs Input

In the Edge Delta App, Cloudwatch Event Logs is now a supported input.

The Cloudwatch Event Logs input type allows you to specify a set of AWS CloudWatch Log Events for Edge Delta to monitor. With this input, you can monitor multiple regions and log streams.

In the app, you can use the visual editor or YAML file to add Cloudwatch Event Logs to a configuration.

To learn more, see Inputs.


December 1, 2021 - Big Panda Integration

In the Edge Delta App, Big Panda is now a supported triggering output.

The Big Panda output will stream notifications and alerts to a specified BigPanda endpoint. In the app, you can use the visual editor or YAML file to add Big Panda to a configuration.

To learn more, see Review Parameters for Trigger Outputs and Integrations.


December 1, 2021 - Loki Integration 

In the Edge Delta App, Loki is now a supported streaming output.

The Loki output streams analytics and insights to your Loki endpoint. In the app, you can use the visual editor or YAML file to add Loki to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


December 1, 2021 - Logz.io Integration

In the Edge Delta App, Logz.io is now a supported streaming output.

The Logz.io output will stream analytics and insights to your Logz.io endpoint. In the app, you can use the visual editor or YAML file to add VictorOps to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


November 30, 2021 - Child Configurations

In the Edge Delta App, you can add a configuration into another, existing configuration. With this action, the existing configurations will convert into a parent and child configuration. 

To learn more, review the Create and Add a Child Configuration section of the Cloud Configuration Backend (CCB) document.


November 30, 2021 - New Compression and Encoding Settings

In the Edge Delta App, you can change compression and encoding settings for Outputs - Archives.

  • For encoding, Edge Delta now supports Parquet.
  • For compression, Edge Delta now supports zstd and Snappy.

Previously, Edge Delta only offered gzip compression and JSON encoding.

To update these settings, you must access the configuration's YAML file. Review the following sample configuration: 

archives:
    - name: my-minio
      type: minio
      access_key: my_access_key_123
      secret_key: my_secret_key_123
      endpoint: play.minio.com:9000
      bucket: ed-test-bucket-minio
      disable_ssl: true
      # Force archive destination to use {endpoint}/{bucket} format instead of {bucket}.{endpoint}/ when reaching buckets.
      s3_force_path_style: true
      encoding: parquet # supported ones: json, parquet
      compression: zstd # supported ones: gzip, zstd, snappy, uncompressed

November 23, 2021 - EDPort Integration

The EDPort streaming integration has been updated to offer JSON as a schema option for sending data.

To learn more, see Review Parameters for Stream Outputs and Integrations.


November 22, 2021 - Local Storage 

In the Edge Delta App, Local Storage is now a supported archiving output.

The Local Storage output will send logs to a file on your local machine. 

To learn more, see Review Parameters for Archive Outputs and Integrations.


November 18, 2021 - Azure Event Hubs

In the Edge Delta App, Azure Event Hubs is now a supported streaming output.

The Azure Event Hubs output will stream analytics and insights to an Azure Event Hubs endpoint. In the app, you can use a YAML file to add Azure Event Hubs to a configuration.

To learn more, see Review Parameters for Stream Outputs and Integrations.


November 12, 2021 - Azure Event Hubs

In the Edge Delta App, Azure Event Hubs is now a supported triggering output.

The Azure Event Hubs output will stream notifications and alerts to a specified Event Hub URL. In the app, you can use the visual editor or YAML file to add Azure Event Hubs to a configuration.

To learn more, see Review Parameters for Trigger Outputs and Integrations.


November 9, 2021 - Create a Token

In the Edge Delta App, you can create a token to give your users specific access to the Edge Delta API system.

Add_Token_Name.png

With tokens, you can specify read / write access for specific backend functionality for your users. In other words, you can create a token to give your users specific read / write access to the Edge Delta API system.

Add_Token_Permissions.png

To learn more, see Tokens.


Share this document