readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_MODE is u-ctags and TAG_OUTPUT_FILESEP is slash #3599

masatake · 2022-12-17T19:43:36Z

Close #3577.
Close #3559 ~~except documentation~~.

codecov · 2022-12-17T19:48:26Z

Codecov Report

Base: 82.88% // Head: 82.87% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (61bb712) compared to base (3ec38c9).
Patch coverage: 76.19% of modified lines in pull request are covered.

❗ Current head 61bb712 differs from pull request most recent head f72f908. Consider uploading reports for the commit f72f908 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3599      +/-   ##
==========================================
- Coverage   82.88%   82.87%   -0.01%     
==========================================
  Files         223      223              
  Lines       54028    54415     +387     
==========================================
+ Hits        44779    45096     +317     
- Misses       9249     9319      +70

Impacted Files	Coverage Δ
optlib/cmake.c	`0.00% <0.00%> (ø)`
optlib/ctags-optlib.c	`0.00% <0.00%> (ø)`
optlib/elixir.c	`0.00% <0.00%> (ø)`
optlib/gdbinit.c	`0.00% <0.00%> (ø)`
optlib/gperf.c	`0.00% <0.00%> (ø)`
optlib/iPythonCell.c	`0.00% <0.00%> (ø)`
optlib/inko.c	`0.00% <0.00%> (ø)`
optlib/kconfig.c	`0.00% <0.00%> (ø)`
optlib/lex.c	`0.00% <0.00%> (ø)`
optlib/man.c	`0.00% <0.00%> (ø)`
... and 50 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

masatake · 2022-12-17T22:01:35Z

#3601 and #3600 must be fixed together in this pull request.

masatake · 2022-12-18T08:05:27Z

The man pages we have to update:

diff --git a/man/ctags-client-tools.7.rst.in b/man/ctags-client-tools.7.rst.in
index 0c086fa89..f0a75f807 100644
--- a/man/ctags-client-tools.7.rst.in
+++ b/man/ctags-client-tools.7.rst.in
@@ -297,6 +297,8 @@ for using notable ones.
        find "input.c" from the file system. In that case,
        ``TAG_PROC_CWD`` gives the tool a hint; "input.c" may be at "/tmp".
 
+       <TODO:input>
+
 ``TAG_PROGRAM_NAME``
        Indicates the name of program generating this tags file.
 
diff --git a/man/readtags.1.rst.in b/man/readtags.1.rst.in
index 5ea49350f..8a8cd87db 100644
--- a/man/readtags.1.rst.in
+++ b/man/readtags.1.rst.in
@@ -91,6 +91,7 @@ than the pattern field. By default, readtags translates them to make it
 human-readable, but when utilizing readtags output in a script or a client
 tool, ``-E`` option should be used. See ctags-client-tools(7) for more
 discussion on this.
+<TODO:input>
 
 Filtering, Sorting, and Formatting
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/man/tags.5.rst.in b/man/tags.5.rst.in
index 999f87469..b899d3e7c 100644
--- a/man/tags.5.rst.in
+++ b/man/tags.5.rst.in
@@ -294,7 +294,7 @@ A tagfield has a name, a colon, and a value: "name:value".
     not handled in the above "value" rules.
 
   EXCEPTION: Universal Ctags allows all these escape sequences in {tagname}
-  also.
+  also. <TODO:input>
 
   - The leading space (0x20) and ``!`` (0x21) in {tagname} are converted
     to ``\x`` prefixed hexadecimal number (``\x20`` and ``\x21``) if the
@@ -448,6 +448,7 @@ command can be used::
 
 (replace ^I with a real <Tab>).
 
+<TODO:input>
 
 TAG FILE INFORMATION:
 
@@ -522,6 +523,8 @@ Exceptions
 
    .. NOT REVIEWED YET (above item)
 
+<TODO:input>
+
 .. _compat-output:
 
 Compatible output and weakness

masatake · 2022-12-25T06:39:14Z

testing on windows
revising the documentation changes

leleliu008 · 2022-12-25T12:50:40Z

brew install --overwrite python@3.10
brew install --overwrite python@3.11

above two lines both are needed

masatake · 2022-12-25T13:44:28Z

@leleliu008 I tried various things at #3608. However, the testing on MacOS is still failed.
Could you make a pull request?

leleliu008 · 2022-12-25T13:56:57Z

OK, I will

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…tory Signed-off-by: Masatake YAMATO <yamato@redhat.com>

4eee45ec6 Merge pull request universal-ctags#48 from masatake/unescaping-input-field-only-when-slash 15b9503ba Read the input field with unescaping if TAG_OUTPUT_MODE is "u-ctags" and TAG_OUTPUT_FILESEP is "slash" 93e0c73d4 Merge pull request universal-ctags#47 from masatake/unescaping-input-field 687f3d9ad Read the input field with unescaping if TAG_OUTPUT_MODE is "u-ctags" 3517d2e68 Use smaller data types for some members in tagFile 93bdb3226 Add a helper function for unescaping 3fb676fde Make macros for comparing field values reusable git-subtree-dir: libreadtags git-subtree-split: 4eee45ec6f4fedb506a9a14b33e97904ec1f09d8

…ng-input-field-3

masatake · 2022-12-26T06:05:50Z

 	--- ./Tmain/ptag-proc-cwd.d/stdout-expected.txt	2022-12-26 01:45:30.827417300 +0000
	+++ D:/a/ctags/ctags/Tmain/ptag-proc-cwd.d/stdout-actual.txt	2022-12-26 01:48:55.102740100 +0000
	@@ -3 +3 @@
	-!_TAG_PROC_CWD	/tmp/ctags-\t-tmain/	//
	+!_TAG_PROC_CWD	/tmp/ctags--tmain/	//

I could not reproduce this on Mingw/Msys running on my local PC.

masatake · 2022-12-29T21:32:58Z

man/tags.5.rst.in

@@ -515,6 +521,14 @@ Exceptions
   exceptions. See {tagname} of Proposal section for more detail about the
   conversion.

+#. {tagfile} in tags file generated by Universal Ctags may contain


This item is newly added as suggested by @AmaiKinono.

masatake · 2022-12-29T21:39:15Z

docs/man/readtags.1.rst

@@ -92,6 +92,13 @@ human-readable, but when utilizing readtags output in a script or a client
 tool, ``-E`` option should be used. See :ref:`ctags-client-tools(7) <ctags-client-tools(7)>` for more
 discussion on this.

+About printing input fields ({tagfile} in :ref:`tags(5) <tags(5)>`) with ``-E`` option:


This is the updated description.

AmaiKinono · 2022-12-30T10:16:41Z

man/readtags.1.rst.in

+when ``TAG_OUTPUT_MODE`` pseudo tag has "u-ctags" and
+``TAG_OUTPUT_FILESEP`` has "slash" as values for their input fields.
+If this condition is not satisfied, readtags prints the fields with
+unescaping. About the pseudo tags, see ctags-client-tools(7).


I think I understand most of the things now, but this is still a bit confusing: If the condition is not satisfied, then according to tags(5), the input field won't contain any escape sequences, right? then we shouldn't need to "print with unescaping" as there are no escape sequences to unescape.

You are correct. I will revise this.
Surprisingly, readtags built with this pull request doesn't behave as I wrote here.
I must update this pull request carefully.

Again, You are correct. After drawing the figure, I understand readtags print the input fields as-is when -E option is given. It doesn't depend on the pseudo tags.
However, the behavior of libreadtags depends on the pseudo tags.

masatake · 2022-12-30T16:58:10Z

As I was afraid, I don't understand what I was doing.

So I draw figures what I wanted.

masatake · 2022-12-30T18:47:09Z

@AmaiKinono updated again.

91d65e4 is taken from your review comment. So we are ready to merge this pull request.

…ODE is u-ctags and TAG_OUTPUT_FILESEP is slash Close universal-ctags#3577. Close universal-ctags#3559 except documentation. Signed-off-by: Masatake YAMATO <yamato@redhat.com>

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…rmat Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…o tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…ctly Signed-off-by: Masatake YAMATO <yamato@redhat.com>

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

/ in pattern fields was converted to \/ in json format. This conversion comes from the format of tags(5). In the tags format, / in pattern fields is used as a separater of the field like: /int main(/ So a / in the middle of the field must be converted to \/. However, in the json format doesn't use such a separater. The conversion is not needed. NOTE: in a regular tag, the conversion is still needed even in the json format because / and ? are used as not only separaters but also indicators of searching direction like: /int main(/ ?int main(? Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…put fields of psuedo tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

…ular tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

… to input fields Co-authored-by: AmaiKinono <amaikinono@gmail.com> Signed-off-by: Masatake YAMATO <yamato@redhat.com>

masatake added the next-release label Dec 17, 2022

masatake added this to the 6.1 milestone Dec 17, 2022

masatake force-pushed the unescaping-input-field branch 2 times, most recently from 5947466 to ae28268 Compare December 25, 2022 06:34

masatake marked this pull request as ready for review December 25, 2022 06:35

masatake requested review from AmaiKinono and removed request for AmaiKinono December 25, 2022 07:01

masatake force-pushed the unescaping-input-field branch 2 times, most recently from 35fa3fa to 3bd06ed Compare December 25, 2022 12:28

masatake added 4 commits December 26, 2022 04:15

docs(web): fix a typo

3547094

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(web): don't refer to a test file already deleted from our reposi…

164bbdc

…tory Signed-off-by: Masatake YAMATO <yamato@redhat.com>

Merge commit '8020dc46f2e41d1891d54eb95838bc7ae63452fb' into unescapi…

e3305ac

…ng-input-field-3

masatake force-pushed the unescaping-input-field branch 9 times, most recently from e49b63f to 18c218b Compare December 26, 2022 01:40

masatake commented Dec 29, 2022

View reviewed changes

masatake force-pushed the unescaping-input-field branch 2 times, most recently from 02f1416 to ec5ee24 Compare December 29, 2022 21:38

masatake commented Dec 29, 2022

View reviewed changes

AmaiKinono reviewed Dec 30, 2022

View reviewed changes

readtags-cmd,comment: delete some comments

52ad100

masatake force-pushed the unescaping-input-field branch 2 times, most recently from 5e5f9f3 to 91d65e4 Compare December 30, 2022 18:43

masatake added 13 commits December 31, 2022 06:14

readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_M…

3ffae71

…ODE is u-ctags and TAG_OUTPUT_FILESEP is slash Close universal-ctags#3577. Close universal-ctags#3559 except documentation. Signed-off-by: Masatake YAMATO <yamato@redhat.com>

readtags,refactor: reduce local variables

852aaeb

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): fix wrong English usage

09e093d

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): update the link for the page describing the JSON output fo…

f1615ba

…rmat Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): write about TAGS_OUTPUT_FILESEP and TAGS_OUTPUT_MODE pseud…

9e7aba1

…o tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

main: verify characters that can be used in a language name more stri…

bc86a58

…ctly Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): make the section headers uppercase

94f1e7e

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

writer-json: don't do tags(5) stype escaping-process when printing in…

c351959

…put fields of psuedo tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

writer-ctags: escape input fields of pseudo tags as same as do in reg…

35614c8

…ular tags Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): use the term "MS Windows" instead of just "Windows"

a0d659c

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(man): fix an indentation

1826019

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

docs(web): add figures illustrating escaping/unescaping input fields

4758655

Signed-off-by: Masatake YAMATO <yamato@redhat.com>

masatake force-pushed the unescaping-input-field branch from 91d65e4 to 61bb712 Compare December 30, 2022 21:14

docs(man): write about the conditions when escaping rules are applied…

f72f908

… to input fields Co-authored-by: AmaiKinono <amaikinono@gmail.com> Signed-off-by: Masatake YAMATO <yamato@redhat.com>

masatake force-pushed the unescaping-input-field branch from 61bb712 to f72f908 Compare December 31, 2022 19:58

masatake merged commit c89b92d into universal-ctags:master Dec 31, 2022

This was referenced Dec 31, 2022

main: escape illegal characters in ptags as explained in tags(5) #3601

Closed

main,win32: reflect useSlashAsFilenameSeparator to the value for TAG_PROC_CWD ptag #3600

Closed

docs(man)/tags(5): write more about {tagfile} #3584

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_MODE is u-ctags and TAG_OUTPUT_FILESEP is slash #3599

readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_MODE is u-ctags and TAG_OUTPUT_FILESEP is slash #3599

masatake commented Dec 17, 2022 •

edited

Loading

codecov bot commented Dec 17, 2022 •

edited

Loading

masatake commented Dec 17, 2022

masatake commented Dec 18, 2022

masatake commented Dec 25, 2022

leleliu008 commented Dec 25, 2022

masatake commented Dec 25, 2022

leleliu008 commented Dec 25, 2022

masatake commented Dec 26, 2022 •

edited

Loading

masatake Dec 29, 2022

masatake Dec 29, 2022

AmaiKinono Dec 30, 2022

masatake Dec 30, 2022 •

edited

Loading

masatake Dec 30, 2022

masatake commented Dec 30, 2022

masatake commented Dec 30, 2022

readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_MODE is u-ctags and TAG_OUTPUT_FILESEP is slash #3599

readtags: unescape input field (a.k.a {tagfile}) only if TAG_OUTPUT_MODE is u-ctags and TAG_OUTPUT_FILESEP is slash #3599

Conversation

masatake commented Dec 17, 2022 • edited Loading

codecov bot commented Dec 17, 2022 • edited Loading

Codecov Report

masatake commented Dec 17, 2022

masatake commented Dec 18, 2022

masatake commented Dec 25, 2022

leleliu008 commented Dec 25, 2022

masatake commented Dec 25, 2022

leleliu008 commented Dec 25, 2022

masatake commented Dec 26, 2022 • edited Loading

masatake Dec 29, 2022

Choose a reason for hiding this comment

masatake Dec 29, 2022

Choose a reason for hiding this comment

AmaiKinono Dec 30, 2022

Choose a reason for hiding this comment

masatake Dec 30, 2022 • edited Loading

Choose a reason for hiding this comment

masatake Dec 30, 2022

Choose a reason for hiding this comment

masatake commented Dec 30, 2022

masatake commented Dec 30, 2022

masatake commented Dec 17, 2022 •

edited

Loading

codecov bot commented Dec 17, 2022 •

edited

Loading

masatake commented Dec 26, 2022 •

edited

Loading

masatake Dec 30, 2022 •

edited

Loading