antidebug_antivm.yar & EMAIL_Cryptowall.yar crashes ClamAV 0.100 on Solaris

[awatkins1966]

Hi,
Has anyone getting the same.

If EMAIL_Cryptowall.yar & antidebug_antivm.yar are used I get core dump on clamav 0.100. Previous versions gave errors but never crashed.

$ /usr/local/clamav0100/bin/clamscan   -r /home/andrew/public/mdefang-l8HAMKFh010488/Work
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/winnow_malware.yara line 84 duplicate identifier "CryptoWall_Resume_phish"
LibClamAV Warning: cli_loadyara: failed to parse or load 1 yara rules from file /usr/local/clamav0100/share/clamav/winnow_malware.yara, successfully loaded 8 rules.
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 497 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 512 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 528 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 544 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 557 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 603 undefined identifier "pe"
LibClamAV Error: yyerror(): /usr/local/clamav0100/share/clamav/antidebug_antivm.yar line 614 undefined identifier "pe"
LibClamAV Warning: cli_loadyara: failed to parse or load 7 yara rules from file /usr/local/clamav0100/share/clamav/antidebug_antivm.yar, successfully loaded 92 rules.
/home/andrew/public/mdefang-l8HAMKFh010488/Work/e/WormVirus: Win.Worm.Mydoom-90 FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/WormVirus: Win.Worm.Mydoom-90 FOUND
Assertion failed: sp == 0, file yara_exec.c, line 177
Abort (core dumped)

Jusr proof it works without these 2 files:

$ cd /usr/local/clamav0100/share/clamav/
$ rm antidebug_antivm.yar EMAIL_Cryptowall.yar

$ /usr/local/clamav0100/bin/clamscan   -r /home/andrew/public/mdefang-l8HAMKFh010488/Work
/home/andrew/public/mdefang-l8HAMKFh010488/Work/e/WormVirus: Win.Worm.Mydoom-90 FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/WormVirus: Win.Worm.Mydoom-90 FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/virus.zip: OK
/home/andrew/public/mdefang-l8HAMKFh010488/Work/WormVirus.zip: Win.Worm.Mydoom-90 FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/PhishingMail: Heuristics.Phishing.Email.SpoofedDomain FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/PhishingMail: Heuristics.Phishing.Email.SpoofedDomain FOUND
/home/andrew/public/mdefang-l8HAMKFh010488/Work/PhishingMail: Sanesecurity.Phishing.Cur.835.UNOFFICIAL FOUND

----------- SCAN SUMMARY -----------
Known viruses: 10619139
Engine version: 0.100.0
Scanned directories: 2
Scanned files: 5
Infected files: 4
Data scanned: 0.11 MB
Data read: 0.11 MB (ratio 1.07:1)
Time: 39.081 sec (0 m 39 s)

Any comments.

Cheers
Andrew

[Warter21]

It is the same on Linux (Slackware).

[amishmm]

same on Arch linux - clamd fails with
clamd[1893]: clamd: yara_exec.c:177: yr_execute_code: Assertion `sp == 0' failed.

Since clamav 0.100 (which I updated today)

[vladki77]

Same on debian 8 after last update (0.99 -> 0.100).
There were other warnings/errors about broken yara rules even before, but none of them fatal.

[Whichcraft]

can confirm for Debian Jessie.

libclamav7:amd64 (0.99.2+dfsg-0+deb8u3, 0.100.0+dfsg-0+deb8u1)
clamav-daemon:amd64 (0.99.2+dfsg-0+deb8u3, 0.100.0+dfsg-0+deb8u1)
clamav-freshclam:amd64 (0.99.2+dfsg-0+deb8u3, 0.100.0+dfsg-0+deb8u1)

syslog:

Jun 25 19:12:57 mail amavis[3777]: (03777-16) (!)ClamAV-clamd: Empty result from /var/run/clamav/clamd.ctl, retrying (2)
Jun 25 19:12:57 mail amavis[3774]: (03774-17) (!)ClamAV-clamd: Empty result from /var/run/clamav/clamd.ctl, retrying (2)
Jun 25 19:12:57 mail amavis[3775]: (03775-14) (!)ClamAV-clamd: Empty result from /var/run/clamav/clamd.ctl, retrying (2)
Jun 25 19:12:57 mail amavis[3778]: (03778-12) (!)ClamAV-clamd: Empty result from /var/run/clamav/clamd.ctl, retrying (2)
Jun 25 19:13:14 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/maldoc_somerules.yar line 235 undefined identifier "uint32be"
Jun 25 19:13:14 mail clamd[11406]: LibClamAV Warning: cli_loadyara: failed to parse or load 1 yara rules from file /var/lib/clamav/maldoc_somerules.yar, successfully loaded 14 rules.
Jun 25 19:13:14 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/packer.yar line 103 undefined identifier "pe"
[...]
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/packer.yar line 20171 undefined identifier "pe"
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: cli_ac_addsig: Can't find a static subpattern of length 2
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: cli_parse_add(): Problem adding signature (3).
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: load_oneyara[verify]: recovered from database loading error
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: load_oneyara[verify]: string failed test insertion: $a0
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: load_oneyara: clamav cannot support 1 input strings, skipping YARA.AsCryptv01SToRM1
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: cli_loadyara: problem parsing yara file /var/lib/clamav/packer.yar, yara rule AsCryptv01SToRM1
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: parse_yara_hex_string: Single byte subpatterns unsupported in ClamAV
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: load_oneyara: error in parsing yara hex string
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: load_oneyara: clamav cannot support 1 input strings, skipping YARA.winrar_sfx
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: cli_loadyara: problem parsing yara file /var/lib/clamav/packer.yar, yara rule winrar_sfx
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: parse_yara_hex_string: Single byte subpatterns unsupported in ClamAV
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Error: load_oneyara: error in parsing yara hex string
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: load_oneyara: clamav cannot support 1 input strings, skipping YARA.mew_11_xx
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: cli_loadyara: problem parsing yara file /var/lib/clamav/packer.yar, yara rule mew_11_xx
Jun 25 19:13:17 mail clamd[11406]: LibClamAV Warning: cli_loadyara: failed to parse or load 1398 yara rules from file /var/lib/clamav/packer.yar, successfully loaded 265 rules.
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 497 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 512 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 528 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 544 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 557 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 603 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 614 undefined identifier "pe"
Jun 25 19:13:18 mail clamd[11406]: LibClamAV Warning: cli_loadyara: failed to parse or load 7 yara rules from file /var/lib/clamav/antidebug_antivm.yar, successfully loaded 92 rules.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: Global size limit set to 104857600 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: File size limit set to 26214400 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: Recursion level limit set to 10.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: Files limit set to 10000.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxEmbeddedPE limit set to 10485760 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxHTMLNormalize limit set to 10485760 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxHTMLNoTags limit set to 2097152 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxScriptNormalize limit set to 5242880 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxZipTypeRcg limit set to 1048576 bytes.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxPartitions limit set to 50.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxIconsPE limit set to 100.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: MaxRecHWP3 limit set to 16.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: PCREMatchLimit limit set to 10000.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: PCRERecMatchLimit limit set to 5000.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Limits: PCREMaxFileSize limit set to 26214400.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Archive support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> BlockMax heuristic detection disabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Algorithmic detection enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Portable Executable support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> ELF support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Mail files support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> OLE2 support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> PDF support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> SWF support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> HTML support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> XMLDOCS support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> HWP3 support enabled.
Jun 25 19:13:47 mail clamd[11406]: Mon Jun 25 19:13:47 2018 -> Self checking every 3600 seconds.
Jun 25 19:13:47 mail clamd[11406]: clamd: yara_exec.c:177: yr_execute_code: Assertion `sp == 0' failed.
Jun 25 19:13:47 mail amavis[3775]: (03775-14) (!)ClamAV-clamd av-scanner FAILED: run_av error: Too many retries to talk to /var/run/clamav/clamd.ctl (Empty result from /var/run/clamav/clamd.ctl) at (eval 102) line 613.\n
Jun 25 19:13:47 mail amavis[3778]: (03778-12) (!)ClamAV-clamd av-scanner FAILED: run_av error: Too many retries to talk to /var/run/clamav/clamd.ctl (Empty result from /var/run/clamav/clamd.ctl) at (eval 102) line 613.\n
Jun 25 19:13:47 mail amavis[3774]: (03774-17) (!)ClamAV-clamd av-scanner FAILED: run_av error: Too many retries to talk to /var/run/clamav/clamd.ctl (Empty result from /var/run/clamav/clamd.ctl) at (eval 102) line 613.\n
Jun 25 19:13:47 mail amavis[3777]: (03777-16) (!)ClamAV-clamd av-scanner FAILED: run_av error: Too many retries to talk to /var/run/clamav/clamd.ctl (Empty result from /var/run/clamav/clamd.ctl) at (eval 102) line 613.\n

[Whichcraft]

downgraded to 0.99.2+dfsg-0+deb8u3 and

apt-mark hold clamav-freshclam clamav-base clamav clamav-daemon

issue currently worked-around.

[vladki77]

It seems that it is enough to disable yara rules, and keep the fresh clamav version:
Set in /etc/clamav-unofficial-sigs/master.conf
yararulesproject_enabled="no"
enable_yararules="no"
And delete *.yar and *.yara from /var/lib/clamav/

[gabviv73]

Is this project still alive ? How could we fix the problem with yara rules ?
Thanks

[enekux]

Hi,
we run into the same issue. The temporal solution provided by @vladki77 helped.
I also ask, how can we fix the problem with Yara rules?
Thank you,

[rephlex]

same problem here,
(14-456 smtpout03) smtpout-03 ~ # cat /etc/*release
CentOS Linux release 7.5.1804 (Core)

(14-456 smtpout03) smtpout-03 ~ # rpm -qa | grep clamav
clamav-server-systemd-0.100.0-2.el7.x86_64
clamav-unofficial-sigs-3.7.2-1.el7.noarch
clamav-data-0.100.0-2.el7.noarch
clamav-0.100.0-2.el7.x86_64
clamav-filesystem-0.100.0-2.el7.noarch
clamav-lib-0.100.0-2.el7.x86_64
clamav-milter-systemd-0.100.0-2.el7.x86_64
clamav-scanner-systemd-0.100.0-2.el7.x86_64
clamav-update-0.100.0-2.el7.x86_64
clamav-milter-0.100.0-2.el7.x86_64

strace says:

[...] blah blah
[pid 8030] mprotect(0x7fbe000ce000, 4096, PROT_READ|PROT_WRITE) = 0
[pid 8030] write(2, "clamd: yara_exec.c:177: yr_execu"..., 69) = 69
[pid 8030] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fbe5bcb6000
[pid 8030] rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
[pid 8030] tgkill(8022, 8030, SIGABRT) = 0
[pid 8030] --- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=8022, si_uid=93} ---
[pid 8031] +++ killed by SIGABRT +++
[pid 8030] +++ killed by SIGABRT +++
[pid 8023] +++ killed by SIGABRT +++
+++ killed by SIGABRT +++

[dominicraf]

It's now a problem in Ubuntu (16.04 and 18.04) too following recent apt-get upgrade.

[lephisto]

Same issue over here... yara rules are an issue as it seems..

[SomePersonSomeWhereInTheWorld]

Fedora 28, same:

ClamAV Error: yyerror(): /var/lib/clamav/antidebug_antivm.yar line 614 undefined identifier "pe"
LibClamAV Warning: cli_loadyara: failed to parse or load 7 yara rules from file /var/lib/clamav/antidebug_antivm.yar, successfully loaded 92 rules.
LibClamAV Error: yyerror(): /var/lib/clamav/maldoc_somerules.yar line 245 undefined identifier "uint32be"
LibClamAV Warning: cli_loadyara: failed to parse or load 1 yara rules from file /var/lib/clamav/maldoc_somerules.yar, successfully loaded 15 rules.

[mmaday]

There's looks to be a bug in the yara rule parsing, which is filed here: https://bugzilla.clamav.net/show_bug.cgi?id=12077 No ETA on a fix. I have removed the yara rules as per @vladki77 's suggestion in #203 (comment) to resolve the issue. According to https://bugzilla.clamav.net/show_bug.cgi?id=12077#c14, the offending yara rule is in Antidebug_AntiVM/antidebug_antivm.yar, so you can be more granular and exclude that to resolve this. If someone has any luck identifying the actual signature, getting a PR/Issue filed at https://github.com/Yara-Rules/rules/issues may be in order.

[extremeshok]

I am currently busy with active development of this.
Full yara support will be re-added along with database validation logic, basically it will not load invalid, broken or unsupported yara rules.

[JB1985]

Same again with v7.2 and Debian stretch.

[Root-Core]

Still the same issue. I've disabled the yara rules for now. Every update enables them though. CC @extremeshok

[hybiepoo]

I found an issue with the winnow_malware.yar file and EMAIL_Cryptowall.yar - they both contained the same identifier. I decided to exclude the winnow file from the sanesecurity sigs by copying the sanesecurity declaration (declare -a sanesecurity_dbs=(... ) into user.conf and commenting out the yar file

#winnow_malware.yara|LOW # detect spam

I deleted the yar file from /var/lib/clamav and all seems to be well now.

[dominicraf]

I found an issue with the winnow_malware.yar file...

Ditto, and this causes clamd to fail entirely. I commented out winnow_malware.yara from master.conf, removed winnow_malware.yara from the clamav database folder and restarted clamd (clamav-daemon). It seems to me that OITC is dead anyway?