Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Without distributed #1854

Closed
wants to merge 31 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
0d8f394
represent unresolved cells using a `Set{Cell}` instead of `Dict`
Pangoraw Oct 25, 2021
a4c6a3d
add several broken macro tests and some fixed too
Pangoraw Oct 25, 2021
5dc100a
unresolve dependencies when defining a new macro
Pangoraw Oct 25, 2021
e925ef1
not distributed
Pangoraw Oct 25, 2021
51c5a84
fix static_resolve
Pangoraw Oct 25, 2021
598575a
fix test
Pangoraw Oct 25, 2021
a5865c3
test #1591
Pangoraw Oct 25, 2021
0116f95
report real timings for macros
Pangoraw Oct 26, 2021
19f6f6e
wrap macro calls in functions
Pangoraw Oct 26, 2021
f31a67e
update comment in `can_be_function_wrapped`
Pangoraw Oct 26, 2021
a5bbe95
remove upper bound in timing test
Pangoraw Oct 26, 2021
70b09f4
Merge remote-tracking branch 'origin/main' into small_update_macro_ru…
Pangoraw Oct 28, 2021
e77142a
use cell UUIDs as computer keys
Pangoraw Oct 28, 2021
18b3517
self updating channel
Pangoraw Oct 28, 2021
43b9092
Macros independent of computers!
dralletje Oct 29, 2021
3348601
Make the "Undefined variable" check in computers a bit less hacky
dralletje Oct 29, 2021
282b839
Make Computers work with `if false; x = 10; end`
dralletje Oct 29, 2021
c8e007d
I KEEP DOING THIS
dralletje Oct 29, 2021
76b3a88
Ah this is what expr_hash is for
dralletje Oct 29, 2021
ae2c9f6
Maybe maybe maybe
dralletje Oct 30, 2021
d673cee
My bad this really is the last
dralletje Oct 30, 2021
80c2af2
GiveMeCellID!!
dralletje Oct 30, 2021
f09c51b
Make sanitize_expr stricter because I can
dralletje Nov 1, 2021
77067f7
Getting started
dralletje Nov 2, 2021
8be0bb2
Trying
dralletje Jan 25, 2022
e6a692c
:shrug:
dralletje Jan 25, 2022
a667938
Merge branch 'main' into without-distributed
dralletje Jan 25, 2022
917ae62
tryin trying trying
dralletje Jan 25, 2022
a158e6b
Was hoping this would give me more insight but mehh
dralletje Jan 25, 2022
945cef2
Update Project.toml
fonsp Jan 26, 2022
c2f5bc7
Merge branch 'main' into without-distributed
dralletje Feb 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
MsgPack = "99f44e22-a591-53d1-9472-aa23ef4bd671"
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
Serialization = "9e88b42a-f829-5b0c-bbe9-9e923198166b"
RelocatableFolders = "05181044-ff0b-4ac5-8273-598c1e38db00"
Sockets = "6462fe0b-24de-5631-8697-dd941f90decc"
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
Expand Down
198 changes: 198 additions & 0 deletions sample/ChildProcesses_benchmark.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
### A Pluto.jl notebook ###
# v0.17.7

using Markdown
using InteractiveUtils

# ╔═╡ 03cbe047-96c6-456f-8adb-8d80390a1742
md"""
# ChildProcesses Benchmark

To run this you must be running ChildProcesses enabled Pluto (so you can `Distributed`).
"""

# ╔═╡ 8e626d50-b8a1-4353-9193-9297d91d6352
import Distributed

# ╔═╡ 808241a9-c7ea-4ca0-9a8b-74658c50a48a
import BenchmarkTools

# ╔═╡ 9b582916-0fd7-4024-9877-1014417c545e
# Act like you a package, quick!
ChildProcesses = @eval Main module ChildProcesses2
include("../src/evaluation/ChildProcessesNotebook.jl")
end

# ╔═╡ f2e2a5de-57dd-4ae7-9860-efa693ef786a


# ╔═╡ 7a3e1cc4-b188-42f2-96c7-931f20ccfa34
expr_to_test = quote
$(fill(1, (1000, 1000)))
end

# ╔═╡ 1326fab3-c8ef-4b0d-a17e-36cbcb7a5275
md"## Distributed.remotecall_eval"

# ╔═╡ 28be2c7a-e2b2-4666-a367-062e3f72ec77
function distributed_benchmark(expr_to_test)
(pid,) = Distributed.addprocs(1)

benchmark = BenchmarkTools.@benchmark begin
Distributed.remotecall_eval(Main, [$pid], $expr_to_test)
end

Distributed.rmprocs([pid])
benchmark
end

# ╔═╡ 11635702-a887-4548-a6b8-55a1c7b08ee7
distributed_benchmark(quote $(1) end)

# ╔═╡ d77fb79a-7105-4321-9151-84d9367c12dc
distributed_benchmark(quote $(fill(1, (1000))) end)

# ╔═╡ 3af58ffa-1d51-42b8-b70e-83a160011ede
distributed_benchmark(quote $(fill(1, (1000, 1000))) end)

# ╔═╡ 7bb5fa42-5f7d-4e83-a55c-ed435f4d4746
distributed_benchmark(quote $(fill(1, (1000, 1000, 10))) end)

# ╔═╡ 0a632e40-73b2-4816-b846-089af80f76cc
md"## ChildProcesses.call"

# ╔═╡ ba51da9b-44a0-41e6-bd37-b05eed451906
function childprocess_benchmark(expr_to_test)
process = ChildProcesses.create_child_process()

benchmark = BenchmarkTools.@benchmark begin
$ChildProcesses.call($process, $expr_to_test)
end

kill(process)
benchmark
end

# ╔═╡ f303a2eb-d4ff-438d-b01a-45bc6a62bf87
childprocess_benchmark(quote $(1) end)

# ╔═╡ 8f6bbed5-e259-4841-a9bc-113df6f13676
childprocess_benchmark(quote $(fill(1, (1000))) end)

# ╔═╡ 5c8e1389-94c3-4abc-a9fd-698e7013bcf6
childprocess_benchmark(quote $(fill(1, (1000, 1000))) end)

# ╔═╡ ef907db1-721d-4695-8752-8fe44db1a0a5
childprocess_benchmark(quote $(fill(1, (1000, 1000, 10))) end)

# ╔═╡ 00000000-0000-0000-0000-000000000001
PLUTO_PROJECT_TOML_CONTENTS = """
[deps]
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
Distributed = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[compat]
BenchmarkTools = "~1.2.2"
"""

# ╔═╡ 00000000-0000-0000-0000-000000000002
PLUTO_MANIFEST_TOML_CONTENTS = """
# This file is machine-generated - editing it directly is not advised

[[BenchmarkTools]]
deps = ["JSON", "Logging", "Printf", "Profile", "Statistics", "UUIDs"]
git-tree-sha1 = "940001114a0147b6e4d10624276d56d531dd9b49"
uuid = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
version = "1.2.2"

[[Dates]]
deps = ["Printf"]
uuid = "ade2ca70-3891-5945-98fb-dc099432e06a"

[[Distributed]]
deps = ["Random", "Serialization", "Sockets"]
uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b"

[[JSON]]
deps = ["Dates", "Mmap", "Parsers", "Unicode"]
git-tree-sha1 = "8076680b162ada2a031f707ac7b4953e30667a37"
uuid = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
version = "0.21.2"

[[Libdl]]
uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb"

[[LinearAlgebra]]
deps = ["Libdl"]
uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"

[[Logging]]
uuid = "56ddb016-857b-54e1-b83d-db4d58db5568"

[[Mmap]]
uuid = "a63ad114-7e13-5084-954f-fe012c677804"

[[Parsers]]
deps = ["Dates"]
git-tree-sha1 = "92f91ba9e5941fc781fecf5494ac1da87bdac775"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "2.2.0"

[[Printf]]
deps = ["Unicode"]
uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[[Profile]]
deps = ["Printf"]
uuid = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"

[[Random]]
deps = ["Serialization"]
uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"

[[SHA]]
uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce"

[[Serialization]]
uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b"

[[Sockets]]
uuid = "6462fe0b-24de-5631-8697-dd941f90decc"

[[SparseArrays]]
deps = ["LinearAlgebra", "Random"]
uuid = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"

[[Statistics]]
deps = ["LinearAlgebra", "SparseArrays"]
uuid = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"

[[UUIDs]]
deps = ["Random", "SHA"]
uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[[Unicode]]
uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
"""

# ╔═╡ Cell order:
# ╟─03cbe047-96c6-456f-8adb-8d80390a1742
# ╠═8e626d50-b8a1-4353-9193-9297d91d6352
# ╠═808241a9-c7ea-4ca0-9a8b-74658c50a48a
# ╠═9b582916-0fd7-4024-9877-1014417c545e
# ╟─f2e2a5de-57dd-4ae7-9860-efa693ef786a
# ╠═7a3e1cc4-b188-42f2-96c7-931f20ccfa34
# ╟─1326fab3-c8ef-4b0d-a17e-36cbcb7a5275
# ╟─28be2c7a-e2b2-4666-a367-062e3f72ec77
# ╠═11635702-a887-4548-a6b8-55a1c7b08ee7
# ╠═d77fb79a-7105-4321-9151-84d9367c12dc
# ╠═3af58ffa-1d51-42b8-b70e-83a160011ede
# ╠═7bb5fa42-5f7d-4e83-a55c-ed435f4d4746
# ╟─0a632e40-73b2-4816-b846-089af80f76cc
# ╟─ba51da9b-44a0-41e6-bd37-b05eed451906
# ╠═f303a2eb-d4ff-438d-b01a-45bc6a62bf87
# ╠═8f6bbed5-e259-4841-a9bc-113df6f13676
# ╠═5c8e1389-94c3-4abc-a9fd-698e7013bcf6
# ╠═ef907db1-721d-4695-8752-8fe44db1a0a5
# ╟─00000000-0000-0000-0000-000000000001
# ╟─00000000-0000-0000-0000-000000000002
1 change: 1 addition & 0 deletions src/Pluto.jl
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ const JULIA_VERSION_STR = 'v' * string(VERSION)
include("./notebook/PathHelpers.jl")
include("./notebook/Export.jl")
include("./Configuration.jl")
include("./evaluation/ChildProcesses.jl")

include("./evaluation/Tokens.jl")
include("./evaluation/Throttled.jl")
Expand Down
5 changes: 5 additions & 0 deletions src/evaluation/ChildProcesses.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
ChildProcesses = @eval Main module ChildProcesses

include("./ChildProcessesNotebook.jl")

end
Loading