csvtk filter2 problem when header column names contain "_"

[hongenxu]

Prerequisites

• make sure you're are using the latest version by csvtk version

Version: 0.13.0

• read the usage

Describe your issue

• describe the problem
  if the header names contain "_", like “ExAC_ALL”，

cat test.hg19_multianno.csv |csvtk filter2 -f ' $6 < 0.05 && $7<0.05' works

cat test.hg19_multianno.csv |csvtk filter2 -f ' $1000g2015aug_all < 0.05 && $1000g2015aug_eas<0.05'

doesn't work

• provide a reproducible example

cat test.hg19_multianno.csv

Chr,Start,End,Ref,Alt,1000g2015aug_all,1000g2015aug_eas,ExAC_ALL,ExAC_AFR,ExAC_AMR,ExAC_EAS,ExAC_FIN,ExAC_NFE,ExAC_OTH,ExAC_SAS
1,907540,907540,T,C,0.00499201,0.0238,0.0020,0,8.672e-05,0.0252,0,1.533e-05,0.0011,0.0012
1,1153073,1153073,G,A,0.000399361,0.002,0.0002,0,8.66e-05,0.0021,0,0,0,6.061e-05
1,1273569,1273569,G,C,0.000399361,0.002,0.0002,0,0,0.0025,0,0,0,0
1,1372668,1372668,C,T,0.00698882,0.0347,0.0033,0,0,0.0441,0,0,0.0014,0.0003
1,1455586,1455586,C,T,0.00119808,0.006,0.0009,0,0,0.0087,0,0.0004,0.0013,0.0002
1,1458237,1458237,G,A,0.00758786,0.0377,0.0033,0,0,0.0430,0,0.0002,0.0039,0.0004
1,1572708,1572708,C,A,0.00319489,0.0089,,,,,,,,
1,2440518,2440518,C,T,0.00279553,0.0099,0.0039,0.0005,0.0260,0.0192,0,0.0005,0.0023,0.0008
1,6257863,6257863,G,A,0.00259585,0.0119,0.0013,0.0001,0.0001,0.0130,0.0030,0.0001,0.0037,0.0004

[shenwei356]

is there any error message?

[hongenxu]

no error message.

the second only output header line:
Chr,Start,End,Ref,Alt,1000g2015aug_all,1000g2015aug_eas,ExAC_ALL,ExAC_AFR,ExAC_AMR,ExAC_EAS,ExAC_FIN,ExAC_NFE,ExAC_OTH,ExAC_SAS

[shenwei356]

Hi @hongenxu , thanks for reporting this bug.

Actually, it's not the mater of "_" but column names starting with digits. Anyway, it is a bug, and I just fixed it, v0.14.0-dev2 works.

[VorontsovIE]

Hi @shenwei356. This problem still exists (0.19.1) if column name doesn't start with a digit but contains digits anywhere except as their trailing symbols. That's enough to change 1000g2015aug_all into g2015aug_all to reproduce the problem.

[shenwei356]

Thank you @VorontsovIE , I've fixed it:

• csvtk_windows_amd64.exe.tar.gz
• csvtk_linux_amd64.tar.gz
• csvtk_darwin_amd64.tar.gz

[VorontsovIE]

Wow! That was lightning fast. Thank you a lot!