Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: token is not defined, but mentioned #116666

Closed
Privat33r-dev opened this issue Mar 12, 2024 · 2 comments
Closed

docs: token is not defined, but mentioned #116666

Privat33r-dev opened this issue Mar 12, 2024 · 2 comments
Labels
docs Documentation in the Doc dir

Comments

@Privat33r-dev
Copy link
Contributor

Privat33r-dev commented Mar 12, 2024 •
edited by bedevere-app bot
Loading

The token term is not being explained (even by means of reference), but mentioned in the chapters 8 and 14 of the tutorial, maybe it can be a good point to add a reference as well?

As a potential solution, we can add another entry in the glossary to shortly provide a basic intuition for a token/code unit. In general, introduction of the term might be of use to explain some concepts, such as error highlighting/handling, syntax. Maybe it would even make sense to introduce it right in the chapter 8 along with error handling.

I would like to see other perspectives on this question.

Originally posted by @Privat33r-dev in #116569 (comment)

Linked PRs
@Eclips4 Eclips4 added the docs Documentation in the Doc dir label Mar 12, 2024
@Almas-Ali
Copy link

I think this needs to be updated. We should add some definition to token. It will be hard to get the proper meaning of this term. We can add 1/2 line to describe what the token term means here. 

@AkashKumarSingh11032001
Copy link

Hi @Privat33r-dev, i think you are right, after going through the following docs, it clear that the word "token" is required to be defined (in case of any beginner try to understand, it would be difficult for them to understand).

encukou added a commit to encukou/cpython that referenced this issue Mar 5, 2025
@encukou
pythongh-116666: Add glossary entry for token
3376144
@bedevere-app bedevere-app bot mentioned this issue Mar 5, 2025
Merged
encukou added a commit that referenced this issue Mar 17, 2025
@encukou @AA-Turner
gh-116666: Add "token" glossary term (GH-130888)
30d5205 
Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 17, 2025
@encukou @AA-Turner @miss-islington
pythongh-116666: Add "token" glossary term (pythonGH-130888)
ab5ea43 
Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)
(cherry picked from commit 30d5205)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Mar 17, 2025
@encukou @AA-Turner @miss-islington
pythongh-116666: Add "token" glossary term (pythonGH-130888)
b43b32a 
Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)
(cherry picked from commit 30d5205)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
This was referenced Mar 17, 2025
Merged
Merged
plashchynski pushed a commit to plashchynski/cpython that referenced this issue Mar 17, 2025
@encukou @AA-Turner @plashchynski
pythongh-116666: Add "token" glossary term (pythonGH-130888)
6e77f5e 
Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
encukou added a commit that referenced this issue Mar 18, 2025
@miss-islington @encukou @AA-Turner
[3.13] gh-116666: Add "token" glossary term (GH-130888) (GH-131367)
25ae204 
gh-116666: Add "token" glossary term (GH-130888)

Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)
(cherry picked from commit 30d5205)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
encukou added a commit that referenced this issue Mar 18, 2025
@miss-islington @encukou @AA-Turner
[3.12] gh-116666: Add "token" glossary term (GH-130888) (GH-131368)
7a8097c 
gh-116666: Add "token" glossary term (GH-130888)

Add glossary entry for `token`, and link to it.
Avoid talking about tokens in the SyntaxError intro (errors.rst); at this point
tokenization is too much of a technical detail. (Even to an advanced reader,
the fact that a *single* token is highlighted isn't too relevant. Also, we don't
need to guarantee that it's a single token.)
(cherry picked from commit 30d5205)

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@encukou encukou closed this as completed Mar 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
docs Documentation in the Doc dir
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@encukou @Privat33r-dev @AkashKumarSingh11032001 @Almas-Ali @Eclips4