I Write Performant Extended Field Extraction Expression

The following tips will enable you to write performant Extended Field Extraction Expression:

  • Ensure that the Extended Field Extraction Expression does not have a Match all regex (.* or \s*\S*) at the start or the end. The Extended Field Extraction Expression as in the example below is not allowed:

    .*(?:[-]*)\s*Call\s+Stack\s+Trace\s*(?:[-]*){callstk:[\s\S]*?}(?:[-]*)\s*Binary.*

    Use the following expression instead:

    (?:[-]*)\s*Call\s+Stack\s+Trace\s*(?:[-]*){callstk:[\s\S]*?}(?:[-]*)\s*Binary
  • Extended Field Extraction Expression must restrict the Match all regex (.* or \s*\S*) usage to 4. See the following example:

    AVDFAlert.*EVENT\S+=\(AN=\"{sefAction:[^"]+}\"\s+AT=\"{sefEndEventTime:[^"]+}\"\s+ASE=\"{sevlvl:[^"]*}\".*URL=\"{detailloc:[^"]*}\".*STN=\"{sefSourceEPName:[^"]*}\".*STT=\"{sefSourceEPType:[^"]*}\".*EN=\"{eventid:[^"]*}\"\s+ET=\"{sefStartEventTime:[^"]+}\".*ES=\"{status:[^"]*}\"\s+CC=\"{sefCommand:[^"]*}\".*UN=\"{sefSourceEPAccountName:[^"]*}\"\s+CHN=\"{sefActorEPName:[^"]*}\"\s+CIP=\"{sefActorEPNwAddress:[^"]*}\".*TOBJ=\"{eventtarget:[^"]*}\".*TTYPE=\"{eventtargettype:[^"]*}\".*TS=\"{sefSourceEPAccountSummaryRisk:[^"]*}\"

    This expression uses .* 10 times, which is not allowed. Break the expression into multiple expressions to ensure that each expression uses .* up to 4 times.

  • The Extended Field Extraction Expression does not use more than 4 conditions or alternatives. See the example below:

    ^\s*\S+\s+:\s+TTY=.*COMMAND=\s*\S*\/(cat|find|ls|more|tail|wc)\s+(-\w+\s+)?{msecrsrcname:\S+}

    This expression is not allowed as it has 6 conditions. Break this expression into 2 expressions as follows, in which case each expression has 3 conditions:

    ^\s*\S+\s+:\s+TTY=.*COMMAND=\s*\S*\/(cat|find|ls)\s+(-\w+\s+)?{msecrsrcname:\S+}
    ^\s*\S+\s+:\s+TTY=.*COMMAND=\s*\S*\/(more|tail|wc)\s+(-\w+\s+)?{msecrsrcname:\S+}
  • Extended Field Extraction Expression has some static text. See the following example:

    (?:POST|PUT|DELETE)\s+[^"]*"\s+(?:-)?(\d+)?\s+{contszin:\d+}

    This expression does not have any static text. Ensure that the expression has at least some minimum static text, if not more.

If any of these rules are violated, the same would be flagged and would have to be fixed before the Extended Field Extraction Expression can be saved.

Once the Extended Field Extraction Expression is in accordance with the above rules, the Test functionality matches the Example Base Field Content with the expression and report on the match status. The match status can be a success, failure or an error. If its a failure or an error, fix the expression and re-test.