Consistent handling of function calls in consume expressions #3647
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Function calls in `consume` are deemed invalid by the syntax checker. However, they seep in via field accessors on the function call expressions. This change fixes ponylang#3453.
kapilash
changed the title
|
Thank you @kapilash! |
SeanTAllen
added
the
changelog - fixed
|
Hi @kapilash, The changelog - fixed label was added to this pull request; all PRs with a changelog label need to have release notes included as part of the PR. If you haven't added release notes already, please do. Release notes are added by creating a uniquely named file in the The basic format of the release notes (using markdown) should be: Thanks. |
jemc
reviewed
Sep 9, 2020
Co-authored-by: Joe Eli McIlvain <joe.eli.mac@gmail.com>
Co-authored-by: Joe Eli McIlvain <joe.eli.mac@gmail.com>
Co-authored-by: Joe Eli McIlvain <joe.eli.mac@gmail.com>
jemc
approved these changes
Sep 10, 2020
|
@SeanTAllen - do you know what's going on with the "PR / Lint bash, docker, markdown, and yaml" CI job? It seems to be complaining/failing on YAML files that weren't changed in this pull request. |
|
@jemc they run across all files as changes to one can have impacts on others. it would appear that the default rules for the yml linting and line length might have changed out from under us. I'll address in another PR and this can be merged with the failures. |
|
@jemc I forgot to squash this so the history will be a little messy. Sorry. I will refrain from doing any merges while sick in the future. |
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are two additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also make sure they work with the following two examples. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are two additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 is particularly insidious as prior to this change, it had "stable" memory growth based on the number of actors created prior to breaking from the `run` behavior and running garbage collection. The amount of stable memory used, is proportional to the number of actors created before breaking from the behavior (and also allowing the scheduler to run other things). With the current change, we get runaway memory growth that needs to be addressed. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are two additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 is particularly insidious as prior to this change, it had "stable" memory growth based on the number of actors created prior to breaking from the `run` behavior and running garbage collection. The amount of stable memory used, is proportional to the number of actors created before breaking from the behavior (and also allowing the scheduler to run other things). With the current change, we get runaway memory growth that needs to be addressed. Note that changeing the % value from 1_000 to 4 results in stable memory usage. The key difference with this change that now has memory growth is that it bypasses a lot of expensive bookkeeping operations related to pony_create and in the process, far more actors are created during the same period of time than prior to this change. Whether the growth issue is purely related to the additional number of actors created over the same period of time is currently unknown. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3: ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4: ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony ctor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 18, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n >= 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. Prior to this change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. The above change allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The example above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected based on the changed in #3647 as soon as the instance's message queue is empty. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following two examples. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Finally, for anyone who works on improving this in the future, here's an additional test program beyond ones that already exist elsewhere for testing pony programs. This program will create a large number of actors that are orphaned but then send themselves to another actor. This should increase their rc count and keep them from being garbage collected. If the program segfaults, then something has gone wrong and the logic related to orphan actors has been broken. The example currently passes as of this commit. ```pony use "collections" actor Holder let _holding: SetIs[ManyOfMe] = _holding.create() new create() => None be hold(a: ManyOfMe) => _holding.set(a) be ping() => var c: U64 = 1_000 for h in _holding.values() do if c > 0 then h.ping() c = c - 1 end end actor ManyOfMe new create(h: Holder) => h.hold(this) be ping() => None actor Main var n: U64 = 500_000 let holder: Holder new create(env: Env) => holder = Holder run() be run() => while(n > 0 ) do ManyOfMe(holder) n = n - 1 if ((n % 1_000) == 0) then @printf[I32]("%ld\n".cstring(), n) run() holder.ping() break end end ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. In order to support the concept of an "orphan actor", additional changes had to be made to the cycle detector protocol and how other actors interact with it. There are two primary things we can do with an orphan actor to improve the rate of garbage collection when the cycle detector is running. When the cycle detector is running actors are collected by the cycle detector when they have no other actors referring to them and can't receive any new messages or are in a cycle that can't receive additional messages. The examples listed in this PR all had problematic memory usage because, actors were created at a rate that overwhelmed the cycle detector and caused large amount of messages to it to back up and sit in its queue waiting to be processed. This commit addresses the "too many messages" problem by making the following actor GC rule changes for orphan actors. Note, all these rules are when the cycle detector is running. * when an actor finishes being run by a scheduler thread "locally delete" it if: * no other actors hold a reference to the actor * the cycle detector isn't holding a reference to the actor "locally deleting" follows the same message free protocol used when the cycle detector isn't running. * when an actor finishes being run by a scheduler thread, pre-emptively inform the cycle detector that we are blocked if: * no other actors hold a reference to the actor * the cycle detector is holding a reference to the actor by pre-emptively informing the cycle detector that we are blocked, the cycle detector can delete the actor more quickly. This "pre-emptively" notify change was introduced in #3647. In order to support these changes, additional cycle detector protocol changes were made. Previously, every actor upon creation informed the cycle detector of its existence. If we want to allow for local deleting, then this won't work. Every actor was known by the cycle detector. All concept of "actor created" messages have been removed by this change. Instead, actors have a flag FLAG_CD_CONTACTED that is set if the actor has ever sent a message to the cycle detector. Once the flag is set, we know that locally deleting the actor is unsafe and we can fall back on the slower pre-emptively inform strategy. The cycle detector works by periodically sending messages to all actors it knows about and asking them if they are currently blocked as the first step in its "path to actor garbage collection" protocol. As actors no longer inform the cycle detector of their existence on creation, the cycle detector needs a new way to discover actors. The first time an actor becomes logically blocked, it sets itself as blocked and notifies the cycle detector of its existence and that it is blocked. This is done by: * setting the actor as blocked * sending an "i'm blocked" message to the cycle detector * setting FLAG_CD_CONTACTED on the actor The overall effect of these changes is: - The rate of garbage collection for short-lived actors is improved. - Fewer cycle detector related messages are generated. Prior to these change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. #1007 requires the "local delete" functionality from this commit whereas #3647 only provided "pre-emptive informing". The addition of "local delete" allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The from #1007 above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected as the instance's message queue is empty so long as it's rc remains 0. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following examples. Each exercises a different type of pattern that could lead to memory growth or incorrect execution. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Finally, for anyone who works on improving this in the future, here's an additional test program beyond ones that already exist elsewhere for testing pony programs. This program will create a large number of actors that are orphaned but then send themselves to another actor. This should increase their rc count and keep them from being garbage collected. If the program segfaults, then something has gone wrong and the logic related to orphan actors has been broken. The example currently passes as of this commit. ```pony use "collections" actor Holder let _holding: SetIs[ManyOfMe] = _holding.create() new create() => None be hold(a: ManyOfMe) => _holding.set(a) be ping() => var c: U64 = 1_000 for h in _holding.values() do if c > 0 then h.ping() c = c - 1 end end actor ManyOfMe new create(h: Holder) => h.hold(this) be ping() => None actor Main var n: U64 = 500_000 let holder: Holder new create(env: Env) => holder = Holder run() be run() => while(n > 0 ) do ManyOfMe(holder) n = n - 1 if ((n % 1_000) == 0) then @printf[I32]("%ld\n".cstring(), n) run() holder.ping() break end end ``` Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. In order to support the concept of an "orphan actor", additional changes had to be made to the cycle detector protocol and how other actors interact with it. There are two primary things we can do with an orphan actor to improve the rate of garbage collection when the cycle detector is running. When the cycle detector is running actors are collected by the cycle detector when they have no other actors referring to them and can't receive any new messages or are in a cycle that can't receive additional messages. The examples listed in this PR all had problematic memory usage because, actors were created at a rate that overwhelmed the cycle detector and caused large amount of messages to it to back up and sit in its queue waiting to be processed. This commit addresses the "too many messages" problem by making the following actor GC rule changes for orphan actors. Note, all these rules are when the cycle detector is running. * when an actor finishes being run by a scheduler thread "locally delete" it if: * no other actors hold a reference to the actor * the cycle detector isn't holding a reference to the actor "locally deleting" follows the same message free protocol used when the cycle detector isn't running. * when an actor finishes being run by a scheduler thread, pre-emptively inform the cycle detector that we are blocked if: * no other actors hold a reference to the actor * the cycle detector is holding a reference to the actor by pre-emptively informing the cycle detector that we are blocked, the cycle detector can delete the actor more quickly. This "pre-emptively" notify change was introduced in #3647. In order to support these changes, additional cycle detector protocol changes were made. Previously, every actor upon creation informed the cycle detector of its existence. If we want to allow for local deleting, then this won't work. Every actor was known by the cycle detector. All concept of "actor created" messages have been removed by this change. Instead, actors have a flag FLAG_CD_CONTACTED that is set if the actor has ever sent a message to the cycle detector. Once the flag is set, we know that locally deleting the actor is unsafe and we can fall back on the slower pre-emptively inform strategy. The cycle detector works by periodically sending messages to all actors it knows about and asking them if they are currently blocked as the first step in its "path to actor garbage collection" protocol. As actors no longer inform the cycle detector of their existence on creation, the cycle detector needs a new way to discover actors. The first time an actor becomes logically blocked, it sets itself as blocked and notifies the cycle detector of its existence and that it is blocked. This is done by: * setting the actor as blocked * sending an "i'm blocked" message to the cycle detector * setting FLAG_CD_CONTACTED on the actor The overall effect of these changes is: - The rate of garbage collection for short-lived actors is improved. - Fewer cycle detector related messages are generated. Prior to these change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` There are three additional examples that were partially addressed by #3647. #1007 wasn't addressed by #3647 because, the key to the previous fix that Dipin and I came up with was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. #1007 requires the "local delete" functionality from this commit whereas #3647 only provided "pre-emptive informing". The addition of "local delete" allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The from #1007 above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected as the instance's message queue is empty so long as it's rc remains 0. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following examples. Each exercises a different type of pattern that could lead to memory growth or incorrect execution. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Finally, for anyone who works on improving this in the future, here's an additional test program beyond ones that already exist elsewhere for testing pony programs. This program will create a large number of actors that are orphaned but then send themselves to another actor. This should increase their rc count and keep them from being garbage collected. If the program segfaults, then something has gone wrong and the logic related to orphan actors has been broken. The example currently passes as of this commit. ```pony use "collections" actor Holder let _holding: SetIs[ManyOfMe] = _holding.create() new create() => None be hold(a: ManyOfMe) => _holding.set(a) be ping() => var c: U64 = 1_000 for h in _holding.values() do if c > 0 then h.ping() c = c - 1 end end actor ManyOfMe new create(h: Holder) => h.hold(this) be ping() => None actor Main var n: U64 = 500_000 let holder: Holder new create(env: Env) => holder = Holder run() be run() => while(n > 0 ) do ManyOfMe(holder) n = n - 1 if ((n % 1_000) == 0) then @printf[I32]("%ld\n".cstring(), n) run() holder.ping() break end end ``` The code and ideas in this commit, while attributed to me is the result of a group effort featuring ideas and/or code from myself, Joe Eli McIlvain, Dipin Hora, and Sylvan Clebsch. Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. In order to support the concept of an "orphan actor", additional changes had to be made to the cycle detector protocol and how other actors interact with it. There are two primary things we can do with an orphan actor to improve the rate of garbage collection when the cycle detector is running. When the cycle detector is running actors are collected by the cycle detector when they have no other actors referring to them and can't receive any new messages or are in a cycle that can't receive additional messages. The examples listed in this PR all had problematic memory usage because, actors were created at a rate that overwhelmed the cycle detector and caused large amount of messages to it to back up and sit in its queue waiting to be processed. This commit addresses the "too many messages" problem by making the following actor GC rule changes for orphan actors. Note, all these rules are when the cycle detector is running. * when an actor finishes being run by a scheduler thread "locally delete" it if: * no other actors hold a reference to the actor * the cycle detector isn't holding a reference to the actor "locally deleting" follows the same message free protocol used when the cycle detector isn't running. * when an actor finishes being run by a scheduler thread, pre-emptively inform the cycle detector that we are blocked if: * no other actors hold a reference to the actor * the cycle detector is holding a reference to the actor by pre-emptively informing the cycle detector that we are blocked, the cycle detector can delete the actor more quickly. This "pre-emptively" notify change was introduced in #3647. In order to support these changes, additional cycle detector protocol changes were made. Previously, every actor upon creation informed the cycle detector of its existence. If we want to allow for local deleting, then this won't work. Every actor was known by the cycle detector. All concept of "actor created" messages have been removed by this change. Instead, actors have a flag FLAG_CD_CONTACTED that is set if the actor has ever sent a message to the cycle detector. Once the flag is set, we know that locally deleting the actor is unsafe and we can fall back on the slower pre-emptively inform strategy. The cycle detector works by periodically sending messages to all actors it knows about and asking them if they are currently blocked as the first step in its "path to actor garbage collection" protocol. As actors no longer inform the cycle detector of their existence on creation, the cycle detector needs a new way to discover actors. The first time an actor becomes logically blocked, it sets itself as blocked and notifies the cycle detector of its existence and that it is blocked. This is done by: * setting the actor as blocked * sending an "i'm blocked" message to the cycle detector * setting FLAG_CD_CONTACTED on the actor The overall effect of these changes is: - The rate of garbage collection for short-lived actors is improved. - Fewer cycle detector related messages are generated. It should be noted that if an actor is incorrectly identified by the compiler as being an orphan when it is in fact not, the end result would be a segfault. During development of this feature, it was discovered that the following code from the Promise select test would segfault: ```pony let pb = Promise[String] .> next[None]({(s) => h.complete_action(s) }) ``` The issue was that .> wasn't being correctly handled by the `is_result_needed` logic in `expr.c`. It is possibly that other logic within the function is faulty and if segfaults are see after this commit that didn't exist prior to it, then incorrectly identifying an actor as an orphan might be the culprit. Prior to these change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` Dipin and I previously worked on #3647 which added "pre-emptively informing" as detailed earlier. The example above from #1007 wasn't addressed by #3647 because the key to #3647 was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. #1007 requires the "local delete" functionality from this commit whereas #3647 only provided "pre-emptive informing". The addition of "local delete" allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The from #1007 above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected as the instance's message queue is empty so long as it's rc remains 0. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following examples. Each exercises a different type of pattern that could lead to memory growth or incorrect execution. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Finally, for anyone who works on improving this in the future, here's an additional test program beyond ones that already exist elsewhere for testing pony programs. This program will create a large number of actors that are orphaned but then send themselves to another actor. This should increase their rc count and keep them from being garbage collected. If the program segfaults, then something has gone wrong and the logic related to orphan actors has been broken. The example currently passes as of this commit. ```pony use "collections" actor Holder let _holding: SetIs[ManyOfMe] = _holding.create() new create() => None be hold(a: ManyOfMe) => _holding.set(a) be ping() => var c: U64 = 1_000 for h in _holding.values() do if c > 0 then h.ping() c = c - 1 end end actor ManyOfMe new create(h: Holder) => h.hold(this) be ping() => None actor Main var n: U64 = 500_000 let holder: Holder new create(env: Env) => holder = Holder run() be run() => while(n > 0 ) do ManyOfMe(holder) n = n - 1 if ((n % 1_000) == 0) then @printf[I32]("%ld\n".cstring(), n) run() holder.ping() break end end ``` The code and ideas in this commit, while attributed to me is the result of a group effort featuring ideas and/or code from myself, Joe Eli McIlvain, Dipin Hora, and Sylvan Clebsch. Closes #1007
SeanTAllen
added a commit
that referenced
this pull request
Sep 19, 2020
We have a few example programs that create a large number of short-lived actors that can exhibit runaway memory growth. The changes in this commit greatly reduce the memory growth or potentially reduce it to be stable. The primary change here is to use some static analysis at compilation time to determine if an actor is "an orphan". That is, upon creation, has no actors (including it's creator) with references to it. In order to support the concept of an "orphan actor", additional changes had to be made to the cycle detector protocol and how other actors interact with it. There are two primary things we can do with an orphan actor to improve the rate of garbage collection when the cycle detector is running. When the cycle detector is running actors are collected by the cycle detector when they have no other actors referring to them and can't receive any new messages or are in a cycle that can't receive additional messages. The examples listed in this PR all had problematic memory usage because, actors were created at a rate that overwhelmed the cycle detector and caused large amount of messages to it to back up and sit in its queue waiting to be processed. This commit addresses the "too many messages" problem by making the following actor GC rule changes for orphan actors. Note, all these rules are when the cycle detector is running. * when an actor finishes being run by a scheduler thread "locally delete" it if: * no other actors hold a reference to the actor * the cycle detector isn't holding a reference to the actor "locally deleting" follows the same message free protocol used when the cycle detector isn't running. * when an actor finishes being run by a scheduler thread, pre-emptively inform the cycle detector that we are blocked if: * no other actors hold a reference to the actor * the cycle detector is holding a reference to the actor by pre-emptively informing the cycle detector that we are blocked, the cycle detector can delete the actor more quickly. This "pre-emptively" notify change was introduced in #3647. In order to support these changes, additional cycle detector protocol changes were made. Previously, every actor upon creation informed the cycle detector of its existence. If we want to allow for local deleting, then this won't work. Every actor was known by the cycle detector. All concept of "actor created" messages have been removed by this change. Instead, actors have a flag FLAG_CD_CONTACTED that is set if the actor has ever sent a message to the cycle detector. Once the flag is set, we know that locally deleting the actor is unsafe and we can fall back on the slower pre-emptively inform strategy. The cycle detector works by periodically sending messages to all actors it knows about and asking them if they are currently blocked as the first step in its "path to actor garbage collection" protocol. As actors no longer inform the cycle detector of their existence on creation, the cycle detector needs a new way to discover actors. The first time an actor becomes logically blocked, it sets itself as blocked and notifies the cycle detector of its existence and that it is blocked. This is done by: * setting the actor as blocked * sending an "i'm blocked" message to the cycle detector * setting FLAG_CD_CONTACTED on the actor The overall effect of these changes is: - The rate of garbage collection for short-lived actors is improved. - Fewer cycle detector related messages are generated. It should be noted that if an actor is incorrectly identified by the compiler as being an orphan when it is in fact not, the end result would be a segfault. During development of this feature, it was discovered that the following code from the Promise select test would segfault: ```pony let pb = Promise[String] .> next[None]({(s) => h.complete_action(s) }) ``` The issue was that .> wasn't being correctly handled by the `is_result_needed` logic in `expr.c`. It is possibly that other logic within the function is faulty and if segfaults are see after this commit that didn't exist prior to it, then incorrectly identifying an actor as an orphan might be the culprit. Prior to these change, examples such as the one from issue #1007 (listed later) would be unable to be collected due to an edge-case in the cycle detector and the runtime garbage collection algorithms. Issue #1007 was opened with the following code having explosive memory growth: ```pony primitive N fun apply(): U64 => 2_000_000_000 actor Test new create(n: U64) => if n == 0 then return end Test(n - 1) actor Main new create(env: Env) => Test(N()) ``` Dipin and I previously worked on #3647 which added "pre-emptively informing" as detailed earlier. The example above from #1007 wasn't addressed by #3647 because the key to #3647 was that when done running its behaviors, an actor can see if it has no additional messages AND no references to itself and can then tell the cycle detector to skip parts of the CD protocol and garbage collect sooner. #1007 requires the "local delete" functionality from this commit whereas #3647 only provided "pre-emptive informing". The addition of "local delete" allows for the cycle detector to keep up with many cases of generating large amounts of "orphaned" actors. The from #1007 above wasn't addressed by #3647 because of an implementation detail in the ORCA garbage collection protocol; at the time that an instance of Test is done running its create behavior, it doesn't have a reference count of 0. It has a reference count of 256. Not because there are 256 references but because, when an actor is created puts a "fake value" in the rc value such that an actor isn't gc'd prematurely. The value will be set to the correct value once the actor that created it is first traced and will be subsequently updated correctly per ORCA going forward. However, at the time that an instance of Test is finished running its create, that information isn't available. It would be incorrect to say "if rc is 256, I'm blocked and you can gc me". 256 is a perfectly reasonable value for an rc to be in normal usage. This isn't a problem with the changes in this PR as the compiler detects that each instance of Test will be an orphan and immediately sets its rc to 0. This allows it to be garbage collected as the instance's message queue is empty so long as it's rc remains 0. Any changes in the future to address lingering issues with creating large numbers of orphaned actors should also be tested with the following examples. Each exercises a different type of pattern that could lead to memory growth or incorrect execution. Example 2 features reasonably stable memory usage that I have seen from time-to-time, increase rapidly. It should be noted that such an increase is rather infrequent but suggests there are additional problems in the cycle-detector. I suspect said problem is a periodic burst of additional messages to the cycle-detector from actors that can be garbage collected, but I haven't investigated further. ```pony actor Main new create(e: Env) => Spawner.run() actor Spawner var _living_children: U64 = 0 new create() => None be run() => _living_children = _living_children + 1 Spawnee(this).run() be collect() => _living_children = _living_children - 1 run() actor Spawnee let _parent: Spawner new create(parent: Spawner) => _parent = parent be run() => _parent.collect() ``` Example 3 has stable memory growth and given that it won't result in any messages being sent to the cycle detector as we have determined at compile-time that the Foo actor instances are orphaned. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end None ``` Example 4 has the same characteristics as example 3 with the code as of this commit. However, it did exhibit different behavior prior to this commit being fully completed and appears to be a good test candidate for any future changes. ```pony actor Main var n: U64 = 2_000_000_000 new create(e: Env) => run() be run() => while(n > 0 ) do Foo(n) n = n - 1 if ((n % 1_000_000) == 0) then @printf[I32]("%ld\n".cstring(), n) end if ((n % 1_000) == 0) then run() break end end actor Foo new create(n: U64) => None ``` Finally, for anyone who works on improving this in the future, here's an additional test program beyond ones that already exist elsewhere for testing pony programs. This program will create a large number of actors that are orphaned but then send themselves to another actor. This should increase their rc count and keep them from being garbage collected. If the program segfaults, then something has gone wrong and the logic related to orphan actors has been broken. The example currently passes as of this commit. ```pony use "collections" actor Holder let _holding: SetIs[ManyOfMe] = _holding.create() new create() => None be hold(a: ManyOfMe) => _holding.set(a) be ping() => var c: U64 = 1_000 for h in _holding.values() do if c > 0 then h.ping() c = c - 1 end end actor ManyOfMe new create(h: Holder) => h.hold(this) be ping() => None actor Main var n: U64 = 500_000 let holder: Holder new create(env: Env) => holder = Holder run() be run() => while(n > 0 ) do ManyOfMe(holder) n = n - 1 if ((n % 1_000) == 0) then @printf[I32]("%ld\n".cstring(), n) run() holder.ping() break end end ``` The code and ideas in this commit, while attributed to me is the result of a group effort featuring ideas and/or code from myself, Joe Eli McIlvain, Dipin Hora, and Sylvan Clebsch. Closes #1007
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Function calls in
consumeare deemed invalid by the syntax checker.However, they seep in via field accessors on the function call
expressions. This change fixes #3453.