SIGABRT: using (") around macros in format-json type-hinting

[mitzkia]

Trying the Admin Guide example from page: URL

$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")

causes double free.

[syslog-ng console output]

Error parsing affile, syntax error, unexpected LL_IDENTIFIER, expecting ')' in syslog-ng.conf at line 12, column 44:
    template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")\n")
                                           ^^^^^^^^^
syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng
*** Error in `sbin/syslog-ng': double free or corruption (fasttop): 0x0000000000cdf0d0 ***
Aborted (core dumped)

[syslog-ng config file]

@version: 3.7
options {
   on_error("drop-property");
};
source s_file { file("input2" flags(no-parse)) ;};
destination d_file {
    file( "output_file.log"
    template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")\n")
    ); };
log {
    source(s_internal);
    source(s_file);
    destination(d_file);
};

[valgrind output]

==72== Invalid free() / delete / delete[] / realloc()
==72==    at 0x4C29E90: free (vg_replace_malloc.c:473)
==72==    by 0x8646DDC: yydestruct (affile-grammar.y:318)
==72==    by 0x86492F2: affile_parse (affile-grammar.c:3885)
==72==    by 0x4E84CE9: cfg_parser_parse (cfg-parser.h:83)
==72==    by 0x4E85460: plugin_parse_config (plugin.c:217)
==72==    by 0x4E9B8F0: main_parse (cfg-grammar.y:699)
==72==    by 0x4E6169A: cfg_parser_parse (cfg-parser.h:83)
==72==    by 0x4E625F5: cfg_run_parser (cfg.c:420)
==72==    by 0x4E62836: cfg_read_config (cfg.c:492)
==72==    by 0x4E7E388: main_loop_read_and_init_config (mainloop.c:450)
==72==    by 0x401A13: main (main.c:258)
==72==  Address 0x8165fb0 is 0 bytes inside a block of size 16 free'd
==72==    at 0x4C29E90: free (vg_replace_malloc.c:473)
==72==    by 0x8647BD6: affile_parse (affile-grammar.y:488)
==72==    by 0x4E84CE9: cfg_parser_parse (cfg-parser.h:83)
==72==    by 0x4E85460: plugin_parse_config (plugin.c:217)
==72==    by 0x4E9B8F0: main_parse (cfg-grammar.y:699)
==72==    by 0x4E6169A: cfg_parser_parse (cfg-parser.h:83)
==72==    by 0x4E625F5: cfg_run_parser (cfg.c:420)
==72==    by 0x4E62836: cfg_read_config (cfg.c:492)
==72==    by 0x4E7E388: main_loop_read_and_init_config (mainloop.c:450)
==72==    by 0x401A13: main (main.c:258)

The workaround for this example is:

$(format-json date=datetime($UNIXTIME) pid=int64($PID) program=$PROGRAM message=$MESSAGE)

[bazsi]

This is a fallout of the latest grammar destruct changes that @ihrwein and
me prepared. This was I think reverted on the 3.7 branch but I am not sure
if there was a release since the reversion.
On Jan 12, 2016 2:28 PM, "mitzkia" notifications@github.com wrote:

Trying the Admin Guide example from page: URL
https://wwwbalabitcom/documents/syslog-ng-ose-37-guides/en/syslog-ng-ose-guide-admin/html/specifying-data-typeshtml

$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")

causes double free

[syslog-ng console output]
Error parsing affile, syntax error, unexpected LL_IDENTIFIER, expecting
')' in syslog-ngconf at line 12, column 44:
template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID")
program="$PROGRAM" message="$MESSAGE")\n")
^^^^^^^^^
syslog-ng documentation:
http://wwwbalabitcom/support/documentation/?product=syslog-ng
mailing list: https://listsbalabithu/mailman/listinfo/syslog-ng
*** Error in `sbin/syslog-ng': double free or corruption (fasttop):
0x0000000000cdf0d0 ***
Aborted (core dumped)

[syslog-ng config file]

@Version: 37
options {
on_error("drop-property");
};
source s_file { file("input2" flags(no-parse)) ;};
destination d_file {
file( "output_filelog"
template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")\n")
); };
log {
source(s_internal);
source(s_file);
destination(d_file);
};

[valgrind output]

==72== Invalid free() / delete / delete[] / realloc()
==72== at 0x4C29E90: free (vg_replace_mallocc:473)
==72== by 0x8646DDC: yydestruct (affile-grammary:318)
==72== by 0x86492F2: affile_parse (affile-grammarc:3885)
==72== by 0x4E84CE9: cfg_parser_parse (cfg-parserh:83)
==72== by 0x4E85460: plugin_parse_config (pluginc:217)
==72== by 0x4E9B8F0: main_parse (cfg-grammary:699)
==72== by 0x4E6169A: cfg_parser_parse (cfg-parserh:83)
==72== by 0x4E625F5: cfg_run_parser (cfgc:420)
==72== by 0x4E62836: cfg_read_config (cfgc:492)
==72== by 0x4E7E388: main_loop_read_and_init_config (mainloopc:450)
==72== by 0x401A13: main (mainc:258)
==72== Address 0x8165fb0 is 0 bytes inside a block of size 16 free'd
==72== at 0x4C29E90: free (vg_replace_mallocc:473)
==72== by 0x8647BD6: affile_parse (affile-grammary:488)
==72== by 0x4E84CE9: cfg_parser_parse (cfg-parserh:83)
==72== by 0x4E85460: plugin_parse_config (pluginc:217)
==72== by 0x4E9B8F0: main_parse (cfg-grammary:699)
==72== by 0x4E6169A: cfg_parser_parse (cfg-parserh:83)
==72== by 0x4E625F5: cfg_run_parser (cfgc:420)
==72== by 0x4E62836: cfg_read_config (cfgc:492)
==72== by 0x4E7E388: main_loop_read_and_init_config (mainloopc:450)
==72== by 0x401A13: main (mainc:258)

The workaround for this example is:

$(format-json date=datetime($UNIXTIME) pid=int64($PID) program=$PROGRAM message=$MESSAGE)

—
Reply to this email directly or view it on GitHub
#865.

[mitzkia]

I forgot to mention the version which I tested, it was the origin/master branch with the following commit ID,
SHA: 443c201
Now I also tested the bug on origin/3.7/master, but it "works" (didn't get the SIGABRT),

[console output from 3.7/master, SHA: 02faa75]

Error parsing affile, syntax error, unexpected LL_IDENTIFIER, expecting ')' in /micek/syslog-ng.conf at line 12, column 44:
    template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")\n")

[bkil-syslogng]

Could you please add a unit test for this? (It should fail on the errored
version, and pass on a non-failing version)

On Wed, Jan 13, 2016 at 10:03 AM, mitzkia notifications@github.com wrote:

I forgot to mention the version which I tested, it was the origin/master
branch with the following commit ID,
SHA: 443c201
443c201
Now I also tested the bug on origin/3.7/master, but it "works" (didn't get
the SIGABRT),

[console output from 3.7/master, SHA: 02faa75
https://github.com/balabit/syslog-ng/commit/02faa75e7ce0bccc79a9898996e598810edc2239
]

Error parsing affile, syntax error, unexpected LL_IDENTIFIER, expecting ')' in /micek/syslog-ng.conf at line 12, column 44:
template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID") program="$PROGRAM" message="$MESSAGE")\n")

—
Reply to this email directly or view it on GitHub
#865 (comment).

[bazsi]

Not easily as its a double free. I wanted to add unit/integration testing
for the grammar, but I didnt get there. We decided to just revert the patch
for 3.7, but @ihrwein knows more.

I would prefer to finish my work on value pairs performance enhancements
first.
On Jan 13, 2016 9:46 AM, "Tamas Nagy" notifications@github.com wrote:

Could you please add a unit test for this? (It should fail on the errored
version, and pass on a non-failing version)

On Wed, Jan 13, 2016 at 10:03 AM, mitzkia notifications@github.com
wrote:

I forgot to mention the version which I tested, it was the origin/master
branch with the following commit ID,
SHA: 443c201
<
443c201

Now I also tested the bug on origin/3.7/master, but it "works" (didn't
get
the SIGABRT),

[console output from 3.7/master, SHA: 02faa75
<
https://github.com/balabit/syslog-ng/commit/02faa75e7ce0bccc79a9898996e598810edc2239

]

Error parsing affile, syntax error, unexpected LL_IDENTIFIER, expecting
')' in /micek/syslog-ng.conf at line 12, column 44:
template("$(format-json date=datetime("$UNIXTIME") pid=int64("$PID")
program="$PROGRAM" message="$MESSAGE")\n")

—
Reply to this email directly or view it on GitHub
<#865 (comment)
.

—
Reply to this email directly or view it on GitHub
#865 (comment).

[ihrwein]

The patches producing this error are reverted on 3.7/master.

I fixed the issue on the master branch, but it seems like I checked only the modules. I'll do the rest.

[juhaszviktor]

Hi,

I can't reproduce this abort using current master of syslog-ng.
@mitzkia is this issue still valid?