-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
proposal: spec: immutable data #37303
Comments
I don't see the advantage of using a magic comment over using |
If you have: const A = 3
const B int = 3
const var C int = 3
//go:immutable
var D int = 3 you can't take address of A or B but you can in case of C or D. I agree that a new compiler directive is something magic but it does its job and has the advantage that it doesn't introduce changes in the language specification. In my opinion, But of course the |
If I do
const var C int = 1
func f(p *int) { *p++ }
is this permissible?
f(&C)
… On 20 Feb 2020, at 18:30, Embedded Go ***@***.***> wrote:
If you have:
const A = 3
const B int = 3
const var C int = 3
//go:immutable
var D int = 3
you can't take address of A or B but you can in case of C or D.
I agree that a new compiler directive is something magic but it does its job and has the advantage that it doesn't introduce changes in the language specification.
In my opinion, const and const var suggests that the immutability is guaranteed by the language specification and should be ensured at compile time what would be desirable but not required by this proposal.
But of course the const var can also be considered because it can do its job and doesn't introduce any new keyword.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
This code will compile but the write access will be detected at runtime and the program will abort with stack trace (the OS will send SIGSEGV or SIGBUS). |
How will this happen, because the data is in a page marked read only? What about this case const var C = new(int)
func f(p *int) { *p++ } is this permissible ?
|
The immutable data will be placed by compiler in .rodata section which is a part of TEXT segment. The most (all?) of current operating systems make the TEXT segment read-only using read-only pages for it. This was described in the first post. The example code you provided is some corner case of this proposal. Let's show more general example: type S struct {i int}
const var (
A = &S{}
B = new(S)
) The But what about |
What about this example
func f() *int {
const var i int
return &i
}
var n *int
for i := 0; i < 20; i++ {
n = f()
}
*n++
is this permissable?
…On Thu, 20 Feb 2020 at 22:18, Embedded Go ***@***.***> wrote:
The immutable data will be placed by compiler in .rodata section which is
a part of TEXT segment. The most (all?) of current operating systems make
the TEXT segment read-only using read-only pages for it. This was described
in the first post.
The example code you provided is some corner case of this proposal.
Let's show more general example:
type S struct {i int}
const var (
A = &S{}
B = new(S)
)
The A = &S{} definitely should compile and both A and S{} should be
placed in .rodata section.
But what about B = new(S)? Should the compiler treat new(S) as the other
form of &S{0} or simply return an error because there is a function call
on right-hand side? I opt for the second choice.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#37303?email_source=notifications&email_token=AAABYA2AQDXXM5WDSJFS6L3RDZRIVA5CNFSM4KYDCSG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMNPQLA#issuecomment-588970028>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAABYA3IDK54Y6YJE5UJ4GLRDZRIVANCNFSM4KYDCSGQ>
.
|
Yes it is. The write access will be detected at runtime and the program will abort. |
But how? the storage class for these allocations must go on the heap,
not ro-data.
…On Thu, 20 Feb 2020 at 22:24, Embedded Go ***@***.***> wrote:
Yes it is. The write access will be detected at runtime and the program will abort.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
Why it must go on the heap? In case of immutable object it can definitely be placed in read-only data section. The |
Consider this example
var n int
func f() *int {
n++
const var N = &n
return N
}
a, b := f(), f()
are a and b the same address?
…On Thu, 20 Feb 2020 at 22:31, Embedded Go ***@***.***> wrote:
Why it must go on the heap? In case of immutable object it can definitely
be placed in read-only data section. The f() will always return the same
address.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#37303?email_source=notifications&email_token=AAABYAYC22RMB3U62EEMYXDRDZS2ZA5CNFSM4KYDCSG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMNQZ7Q#issuecomment-588975358>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAABYA53RMYJ776RYJBM7ZDRDZS2ZANCNFSM4KYDCSGQ>
.
|
But your examples shows the weakness of the |
I was wrong in the first part of the previous answer. Your example code is valid. The n is a global variable and the relocation on the right hand side can be determined at compile time. So yes a and b are the same address. This proposal in the current form doesn't forbid it. |
If a and b have the same address, are there contents equal? It sounds like you’re proposing C’s static storage class. I don’t think that is what most people think of when they think immutable nor something which makes a lot of sense in a multi threaded program. |
I don't understood your point in the case of the last example. The compiler can simply optimize out
The proposal is about the data embedded in the code that are intended to be immutable. Nothing more. I mentioned that i don't like the |
After some time with The following code: const var A = &S{1, &C{2}, &n} is a condensed form of the following declaration: const var (
A = &s
s = S{1, &c, &n}
c = C{2}
) It should be read as:
You can use var n int
func f() *S {
const var s = S{1, &n}
return &s
} and it's equivalent of: var n int
const var s = S{1, &n}
func f() *S {
return &s
} but in the first case |
If we are going to permit taking the address of an immutable value, we must prohibit changing that value through to the pointer. Nothing else makes sense. We can't say it "may or may not be checked at runtime." If we can't detect the invalid change at compile time, we must panic at run time. And that means we need some way to implement that efficiently, which seems hard. If we can't take the address of an immutable value, then this is essentially the same as #6386. |
The original proposal postulated using a compiler directive so it made sense to say that the compiler may ignored it without affecting the correct code. I agree that in case of
I think that without introducing something like const vars as function parameters and struct/maps/slice elements the general detection at compile time is impossible. I don't know if I would like to see such big change in the language. My intention was to introduce very lightweight change to the compiler behavior that leaves some data in the read-only section of memory. This allows to use OS and hardware mechanisms to detect invalid changes efficiently just as it's done in the case of nil pointer dereference. |
The proposed One concern here is that some variable initializers must be constructed at run time, such as code like //go:immutable
var V = F() How should that be implemented? I understand the desire to have a simple mechanism to put values into read-only memory, but I think that to be useful it needs to be consistent and simple and completely reliable. I don't yet see how this proposal accomplishes that. |
This proposal disallows this: "The compiler should return an error if the data on the right hand side cannot be generated at the compile time."
Two facts:
Questions:
The solution for Go should be simple, elegant and cheap. That's what this discussion is for. I agree the I've an interest to leave immutable data in the read-only memory because I try to program in Go for systems with very small RAM (< 512 KB). For now I'm dealing with such data in an ugly way at the linker level and for now this is enough to keep things moving forward. I'm fine to live with this for some time in the hope that we will come up with something sensible here. |
I don't like the idea of using There are some compiler directives in place, and some build flags, however they don't change the semantics of the language. |
@deanveloper, if you never actually try write to the variable, then the semantics of an immutable variable are indistinguishable from a mutable one. (That is: if the program is correct, then the semantics don't change — the comment only changes the behavior of programs that are not correct to begin with.) |
I guess that my opinion is more just that a comment should not dictate whether a program is able to compile or not, whether it is "correct" or not. ie:
This does not compile because of a comment in the code. If we want immutability in Go, then we should introduce it to the language itself, not only into the compiler. People shouldn't have to use a compiler directive just to achieve immutability. At a high level, people should be able to remove all comments from a program and have it still function the same. At the low level, some compiler directives may be needed in order for a program to function properly. However, I would argue that |
It will be compiled. It will fail at runtime if the hardware will detect the write to the read only memory. |
Let's forget the syntax for now. There is alternate syntax proposed without a magic comment: const var x = 5 The question is whether the current lack of the way to make data immutable bothers us at all? |
I was not aware of this, I had assumed that the compilation would fail. My mistake.
It does bother me personally. There have been several instances where I would have liked to define constant structs, slices, etc., which Go of course does not support. My personal solution was to try to introduce constant array/slice/map/struct types, including untyped constants for each "class" of types. Although that proved harder to implement than I originally thought it would once I started writing the proposal. |
This restriction makes this seem very similar to #6386. Perhaps we should close this one as a dup. The main difference seems to be that this clearly permits addressability, whereas #6386 is less clear on that. The main goal of this issue seems to be moving initializers into read-only memory. That in itself does not require a language change. In principle, the toolchain could observe that nothing changes a value, and in that case move the value into read-only memory. This is harder for an exported variable, but not impossible. |
The goal is also to detect invalid changes of data intended to be read-only. Using read-only memory is simple way to do that at runtime. This proposal puts a clear demarcation between constants and read-only variables. The constants remain unaffected. They are simple scalar values with strings as some exception. You can't take address of constant. The read-only variables have no restrictions on the internal structure but there are restrictions when it comes to the way of initialization. You can access them by reference and this is a very important part of this proposal because passing by value can be impractical in case of data large in size. |
I'm not sure that every target that Go supports can reliably put things into read-only memory. Although it may be that the restriction to only constant initializers does permit that. There is clearly an interest in adding some form of immutable data to Go, as there have been several issues opened about it. I'm not sure that restricting immutable data to constant initializers will be satisfactory to everyone who wants the feature. |
If we allow I follow the #6386 and I see you are considering more strict and more complicate solutions. But be open to some nonstandard approach. Does a hint for the compiler and at the same time an information for the programmer isn't all we need? The
It would be very nice to initialize the variable at runtime and make it read-only at any time. But it smells to me like C++++. Does it fit our language? |
I don't think it's acceptable to have |
OK. If you want to address this in a better way I waiting for it. But the current state is:
I think we've exhausted the topic. Maybe someone can add something. If not, this issue can be closed preferably with a link to it and small summary in the #6386 which seems to be the main thread on this topic. |
One option might be to use memory protection on platforms that support it, and best-effort run-time checks on other platforms. For example, “read-only” objects could be tagged with an additional header containing a checksum, which the collector could re-verify during every mark phase: instead of (just) scanning the object for pointers, the collector could also scan it for mutations. The race detector could be similarly employed: a “read-only” object effectively has a “reader” at every possible point in time, which could be modeled as a reader in a background goroutine without any other happens-before relationship. Then, testing the package with the race detector enabled would clearly diagnose the source of the erroneous write. |
I think read-only is necessary in some situation.In my project,the situation in which read only params are changed unknowingly has occurred several times.Simplified code likes below.
In this usual situation,if we call the function "GetMultipleRewards" ,rewardA.Count will be changed unknowingly.It is easy for new golang coder to write this code but it will cost several hours to find the bug.
But if Item has 100 propertis,we have to write 100 get functions.And this is only one struct,if there are 100,200 or more struct,what happens? |
This issue describes language feature proposal to immutable data.
There are more general proposals for Go 2 that postulate changes in the language type system:
Support read-only and practical immutable values in Go
Read-only and immutability
Immutable type qualifier
This proposal isn't as general as the ones mentioned above and focuses only on the data embedded in the code as in the example below (taken from the unicode package):
The problems this proposal tries to solve
If a package exports some data (explicitly or implicitly) that is intended to be immutable there is no way in the current language specification/implementation to ensure immutability or to detect that some faulty code changes the exported data.
In case of microcontroller based embedded systems the mutable data is copied from Flash to RAM at the system startup. In such systems there is a very little RAM because the immutable parts of the program (text and read-only data) are intended to be executed/read by the CPU directly from Flash. There is no way in the current language implementation to leave the immutable data in Flash which causes that the available RAM overflows very quickly as you import more packages.
Language changes
This proposal doesn't require changes to the language specification. It can be implemented by adding a new compiler directive as in the example bellow:
Edit: There is another syntax proposed that requires change in the language specification:
Unlike
const x = 2
theconst var y = 2
allows to take address of y.Implementation
The go:immutable directive should make the variable and any composite literals used to construct it immutable. The compiler should return an error if the data on the right hand side cannot be generated at the compile time. Immutable data should be placed in .rodata section.
The go:immutable directive can be documented as a hint directive that may or may not be implemented by the compiler, the hardware or the operating system.
An immutability violation is detected at runtime and cause the program abort. The detection relies on the operating system which usually uses read-only pages for read-only sections. In case of embedded systems the immutability violation can be detected by hardware and generate an exception.
Design decision argumentation
Using the compiler directive instead of new keyword or an existing keyword combination like
const var
has the advantage that it doesn't introduce any changes to the language specification. If the more general approach for immutability will be developed the directive can be easily removed from the compiler specification.Tests
I've done some tests simulating the go:immutable directive at the linker level by adding the following code to the Link.dodata function:
It moves to the .rodata section all "static temp" symbols from the unicode package that correspond mainly to the composite literals used to initialize global variables. The impact on the code generated for simple Hello, World! program:
is as follow:
before:
after:
As you can see about 48 KB have been moved from the data segment to the text segment and they are all from unicode package only. It isn't impressive from OS capable system point of view but it's a game changer in case of microcontroller based embedded systems which rarely have more than 256 KB of RAM.
Impact on the existing code
Introducing go:immutable directives for immutable data in the standard library and other packages shouldn't affect the correct code in any way. The faulty code can stop work.
Additional explanation
See additional explanation below which is also an example of using
const var
instead of//go:immutable
.The text was updated successfully, but these errors were encountered: