Buffer parser using variables with capture groups.

Tests BufferParser behavior with named capture groups in variable schemas. More...

Functions
void	single_line_with_capture ()
	Validates tokenization behavior when using capture groups in variable schemas.
void	single_line_with_optional_capture ()
	Validates tokenization behavior when using optional capture groups in variable schemas.
void	single_line_with_nested_capture ()
	Validates tokenization behavior when using nested capture groups in variable schemas.
void	multi_capture_one ()
	Tests a multi-capture rule parsing an Android log.
void	multi_capture_two ()
	Tests a multi-capture rule parsing a Kubernetes log.
void	multi_capture_non_unique_names ()
	Tests a multi-capture with non-unique names.
void	multiple_headers ()
	Tests a schema with multiple headers.

Detailed Description

Tests BufferParser behavior with named capture groups in variable schemas.

Verifies:

• Symbol registration for variables and capture groups
• Correct association of tag positions
• Proper assignment and lookup of tag registers

Useful for validating advanced schema features like (?<name>...) integration.

See also

Buffer parser using variables without capture groups. for simpler variable matching.

Function Documentation

multi_capture_non_unique_names()

void multi_capture_non_unique_names

(

)

Tests a multi-capture with non-unique names.

This test verifies that a buffer_parser with multiple capture rules with non-unique capture rules can be generated and used correctly.

Schema Definition

delimiters: \n\r[:,
var1:(?<capture>[A-Za-z]+123) text (?<capture>[A-Za-z]+123)
var2:(?<capture>[A-Za-z]+123) text text

Input Example

"Log is myCapture123 text anotherCapture123 and then another variable is capture123 text text"

Expected Logtype

"Log is <capture> text <capture> and then another variable is <capture> text text"

Expected Tokenization

"Log" -> uncaught string
" is" -> uncaught string
" " -> uncaught string
"myCapture123 text anotherCapture123" -> "var1"
" and" -> uncaught string
" then" -> uncaught string
" another" -> uncaught string
" variable" -> uncaught string
" is" -> uncaught string
" " -> uncaught string
"capture123 text text" -> "var2"

multi_capture_one()

void multi_capture_one

(

)

Tests a multi-capture rule parsing an Android log.

This test verifies that a multi-capture rule correctly identifies the location of each capture group. It tests that BufferParser correctly flattens the logtype, as well as stores the full tree correctly.

Schema Definition

delimiters: \n\r[:,
header:(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}) (?<PID>\d{4}) (?<TID>\d{4})
\
       (?<LogLevel>I|D|E|W)

Input Example

"1999-12-12T01:02:03.456 1234 5678 I MyService A=TEXT B=1.1"

Expected Logtype

"<timestamp> <PID> <TID> <LogLevel> MyService A=TEXT B=1.1"

Expected Tokenization

"1999-12-12T01:02:03.456 1234 5678 I" -> "header"
" MyService" -> uncaught string
" A=TEXT" -> uncaught string
" B=1.1" -> uncaught string

multi_capture_two()

void multi_capture_two

(

)

Tests a multi-capture rule parsing a Kubernetes log.

This test also verifies that a multi-capture rule correctly identifies the location of each capture group. It tests that BufferParser correctly flattens the logtype, as well as stores the full tree correctly.

Schema Definition

delimiters: \n\r[:,
header:(?<timestamp>[A-Za-z]{3} \d{2} \d{2}:\d{2}:\d{2})
ip-(?<IP>\d{3}-\d{2}-\d{2}-\d{2}) \
       ku\[(?<PID>\d{4})\]: (?<LogLevel>I|D|E|W)(?<LID>\d{4}) \
       (?<LTime>\d{2}:\d{2}:\d{2}\.\d{4})    (?<TID>\d{4})

Input Example

"Jan 01 02:03:04 ip-999-99-99-99 ku[1234]: E5678 02:03:04.5678    1111 Y failed"

Expected Logtype

"<timestamp> ip-<IP> ku[<PID>]: <LogLevel><LID> <LTime>    <TID> Y failed"

Expected Tokenization

"Jan 01 02:03:04 ip-999-99-99-99 ku[1234]: E5678 02:03:04.5678    1111" -> "header"
" Y" -> uncaught string
" failed" -> uncaught string

multiple_headers()

void multiple_headers

(

)

Tests a schema with multiple headers.

This test verifies that a buffer_parser with multiple headers can be generated and used correctly.

Schema Definition

delimiters: \n\r[:,
header:[a-z]+ (?<timestamp>[A-Za-z]{3} \d{2} \d{2}:\d{2}:\d{2}) [a-z]+ (?<int>\d+) [a-z]+
header:[a-z]+ (?<timestamp>...) [a-z]+ (?<timestamp>...) [a-z]+
header:[a-z]{4,0}
header:(?<int>\d+) ((?<hex>[a-fA-F]+)|(?<timestamp>...)){0,1} abc:(?<timestamp>...){0,1}
float:\d+\.\d+

Input Example

"text Jan 01 02:03:04 text 123 text word 12.12\na\n"
"text Feb 01 02:03:05 text Mar 29 12:11:10 text word 12.12\na\n"
"text word 12.12\na\n"
"123  abc: 12.12\na\n"
"123 DFF abc: 12.12\na\n"
"123 Dec 10 11:11:11 abc: 12.12\na\n"
"123  abc:Apr 10 11:11:11 12.12\na\n"
"123 DFF abc:May 12 05:06:07 12.12\na\n"
"123 Jun 18 08:12:21 abc:Jul 21 02:11:12 12.12\na"

Expected Logtype

"text <timestamp> text <int> text word <float><newLine>a\n"
"text <timestamp> text <timestamp> text word <float><newLine>a\n"
"text word <float><newLine>a\n"
"<int>  abc: <float><newLine>a\n"
"<int> <hex> abc: <float><newLine>a\n"
"<int> <timestamp> abc: <float><newLine>a\n"
"<int>  abc:<timestamp> <float><newLine>a\n"
"<int> <hex> abc:<timestamp> <float><newLine>a\n"
"<int> <timestamp> abc:<timestamp> <float><newLine>a"

single_line_with_capture()

void single_line_with_capture

(

)

Validates tokenization behavior when using capture groups in variable schemas.

This test verifies the BufferParser's ability to:

• Recognize a variable definition containing a named capture group.
• Identify and register both the variable name and the capture group name as valid symbols.
• Link the capture group to its associated tag IDs and registers.
• Extract matched positions correctly when parsing a token.
• Fail to match tokens that don't align exactly with the specified capture pattern.

Schema Definition

delimiters: \n\r[:,
myVar:userID=(?<uid>123)

Test Input

"userID=123 userID=234 userID=123 123 userID=123"

Expected Logtype

"userID=<uid> userID=234 userID=<uid> 123 userID=<uid>"

Expected Tokenization

"userID=123" -> "myVar" with "123" -> "uid"
" userID=234" -> uncaught string
" userID=123" -> "myVar" with "123" -> "uid"
" 123" -> uncaught string
" userID=123" -> "myVar" with "123" -> "uid"

single_line_with_nested_capture()

void single_line_with_nested_capture

(

)

Validates tokenization behavior when using nested capture groups in variable schemas.

This test is an extension of single_line_with_capture that verifies the correct behaviour when a nested capture groups are not found.

Schema Definition

delimiters: \n\r[:,
myVar:userID=(?<full>abc_(?<uid>\d{3}))

Test Input

"userID=abc_123 userID=abc_456"

Expected Logtype

"userID=<uid> userID=<uid>"

Expected Tokenization

"userID=abc_123" -> "myVar" with "abc_123" -> "full", "123" -> "uid"
" userID=abc_456" -> "myVar" with "abc_456" -> "full", "456" -> "uid"

single_line_with_optional_capture()

void single_line_with_optional_capture

(

)

Validates tokenization behavior when using optional capture groups in variable schemas.

This test is an extension of single_line_with_capture that verifies the correct behaviour when an optional capture group is not found.

Schema Definition

delimiters: \n\r[:,
myVar:userID=(?<uid>123){0,1}

Test Input

"userID=123 userID= userID=456"

Expected Logtype

"userID=<uid> userID= userID=456"

Expected Tokenization

"userID=123" -> "myVar" with "123" -> "uid"
" userID=" -> "myVar" with empty -> "uid"
" userID=456" -> uncaught string