Buffered Trace Filters

George Alpizar
George Alpizar
  • Updated

Overview

This filter type handles trace logs.

  • Edge Delta defines trace log as a set of logs that can be tied together with an ID, such as a trace ID or request ID.

At a high level, this filter type:

  • Groups logs by a specified ID, then
  • Verifies that all relevant events of that trace (or request) is collected, and then
  • Discards or passes the trace logs, based on the configuration.

This filter type offers the following pass options:

  • Pass through failed operation events
  • Pass through high-latency operation events
  • Pass through certain percentage of successful events

Review Parameters

Review the following parameters that you can configure in the Edge Delta App:

YAML Description
name

Enter a descriptive name for this filter.

When you create a workflow, you will use this label to enter your filter into the workflow.

This parameter is required. 

Review the following example:

name: appinsight_trace_filter
type

Enter buffered-trace.

This parameter is required. 

Review the following example:

type: buffered-trace
pattern

Enter a regular expression pattern to define which strings to match on.

This parameter is optional. 

Review the following example:

pattern: "TRACE"
trace_id_pattern

Enter a regular expression pattern to extract the trace ID values from logs. Enter a regex with single capture group.

This parameter is optional. 

Review the following example:

trace_id_pattern: "\"operation_Id\": \"(?P<id>\\w+)\""
failure_pattern

Enter a regular expression pattern to indicate that a match with the trace event (group of logs sharing same ID) is a failure.

Failures are passed through this filter.

This parameter is optional. 

Review the following example:

failure_pattern: \"status\":\"Failed\"
latency_pattern

Enter a regular expression pattern to extract the latency value from the trace logs.

You must enter a regex with a single numeric capture group.

Only 1 of the logs that belongs to the same trace ID should have latency information, or the last log will be picked to represent the latency of the trace.

Once the latency value is extracted and converted to a number, this value can be used in conjunction with the latency_threshold parameter to pass through high-latency traces.

This process is useful to collect the high-latency traces, in addition to the failed traces that already passed throughout, based on the failure_pattern parameter.

This parameter is optional. 

Review the following example:

latency_pattern: "\"latency\": \"(?P<latency>\\d+)\""
latency_threshold

Enter a numeric value to represent the threshold for high-latency limit.

The latency of a trace is extracted with the latency_pattern parameter.

This parameter is optional. 

Review the following example:

latency_threshold: 500.0
success_sample_rate

Enter a number to indicate the percentage of successful traces that you want to receive. You can enter a number between 0 and 1.

The default setting is 0, which means all successful traces are discarded. If you enter 0.2, then 20% of successful traces will pass through the filter. 

Note

Any trace event without a failure_pattern match indicates successful trace. 

This parameter is optional. 

Review the following example:

success_sample_rate: 0.2
trace_deadline

Enter a golang duration string to represent the max duration of a trace. Once the specified trace deadline is reached, the buffered trace filter will take all events that belong to the same trace, apply the filters/sampling, and then pass through the relevant events.

This parameter is optional. 

Review the following example:

trace_deadline: 1m

Review Sample Configuration

Review the following sample configuration: 

filters:
- name: trace_filter
  type: buffered-trace
  trace_id_pattern: \"trace_id\":\"(?P<id>\w+)"
  failure_pattern: \"status\":\"Failed\"
  trace_deadline: 1m
  success_sample_rate: 0.0
  latency_pattern: \"duration\":\"(?P<latency>\d+)\"
  latency_threshold: 8000

 

Share this document