devdocs notes on memory layout and runtime initialisation #9468

samoconnor · 2014-12-27T05:12:37Z

Some notes I made while learning about runtime internals.
Maybe useful for other new developers.
But maybe misleading because eggnog :)

tkelman · 2014-12-27T08:47:32Z

Very cool, thanks for writing these down! For reference, since it touches on related stuff (kinda), here's Jeff's talk from JuliaCon: https://www.youtube.com/watch?v=osdeT-tWjzk

timholy · 2014-12-27T14:59:08Z

I haven't looked at this in any detail whatsoever, but big +1 for doing this!

Keno · 2014-12-27T15:05:12Z

doc/devdocs/init.rst

+Next `parse_opts()
+<https://github.com/JuliaLang/julia/blob/master/ui/repl.c#L80>`_
+is called to process command line options. Note that parse_opts()
+only deals with options that effect early initialisation. Other


how about affect code generation or early initialization?

vtjnash · 2014-12-28T02:33:45Z

doc/devdocs/object.rst

+    }
+
+Note that all objects are allocated in multiples of 8 bytes, so the
+smallest object size is 16 bytes (8 byte type pointer + 8 bytes


the smallest object should be 8 bytes (8 byte type pointer + 0 bytes data)

:)
I guess that's strictly speaking true, but to my mind, an "object" that contains only type information and no data is not really an object, it's just a "meta-object".

it's been given the name singleton and is optimized and used fairly heavily (nothing is a builtin type that uses this). thus it's a pretty important type for understanding codegen.cpp and cgutils.cpp.

Good to know. Thx.
noted in latest commit.

vtjnash · 2014-12-29T00:58:39Z

doc/devdocs/object.rst

+
+.. sidebar:: `special case. <https://github.com/JuliaLang/julia/blob/master/src/jltypes.c#L2973>`_
+
+    The special singleton object :data:`nothing` is "void".


the unique instance of any type without any fields is called a singleton. nothing::Void is an example, not a special-case.

samoconnor · 2015-02-02T06:29:56Z

Hi All,
Is there any interest in merging this PR?
If not I'll close it.
Cheers,
Sam

timholy · 2015-02-02T11:56:39Z

I just took a glance.

One thing I noticed is that your line-number links are not to a specific commit, and will go stale as the code changes (more rapidly than the actual content they describe). One way you can prevent this is by hitting y when you're on the GitHub page with the source: it will generate a URL that includes the sha1 for current master.

Probably the easiest way to make that change would be by search-and-replace with your editor, after you try the y trick once to see how the URL is formed.

samoconnor · 2015-02-02T22:20:04Z

Hi @timholy,
Re line-number links, see comment from Dec 28:
... I admit I was worried about the line-number URL links too as I pasted them into the doc. However, until github has a syntax for linking to a function definition I think it's better than nothing. My guess is that the line numbers will drift over time, but that at least the links will take the reader close enough to the right place that it's still useful. I considered the alternative of using a link to a fixed version of the file, but I decided that it is better that the reader sees the latest code if they are reading about internals. If they find an annoyingly bad link, they can fix it.

timholy · 2015-02-02T22:42:59Z

I guess my point is this: rather than saying (this from your first link),

Execution starts at main in julia/ui/repl.c

it might be better to say

Execution starts at main in julia/ui/repl.c

Those might look the same, but you'll see they already take you to separate places. The fact that I had to update the line numbers is both illustrative and irrelevant: the key thing is that the URL I provided is specific to a particular commit. (In case you can't click on the pencil icon and see the actual URL, it's https://github.com/JuliaLang/julia/blob/2f97c78baae7c4004f2cb0890e0ff521a3fac51a/ui/repl.c#L370-L373. Note the sha1, 2f97c78, refering to a particular commit.) That gives people who might be reading your documentation (1) a sense for which version of julia it was written for, (2) permalinks to code that match the content of what you're actually talking about, and (3) an easy way to compare two versions and see what's changed in the meantime. For someone who's learning this for the first time, it's presumably easier for people see the actual code you were referring to and then for them to find where that code moved to than it is for them to try to guess what used to be sitting at that position.

Like I said, you get such permalinks by hitting y when you're on the source-code page in GitHub. But for you probably the easiest thing would be to figure out which sha1 julia was at when you wrote the documentation, and do a search-replace so that all the links are current.

samoconnor · 2015-02-02T23:19:25Z

I'm quite familiar with making links to a particular commit. In this case I decided not to...

Dec 28: I considered the alternative of using a link to a fixed version of the file, but I decided that it is better that the reader sees the latest code if they are reading about internals. If they find an annoyingly bad link, they can fix it.

timholy: For someone who's learning this for the first time, it's presumably easier for people see the actual code you were referring to and then for them to find where that code moved to than it is for them to try to guess what used to be sitting at that position.

Dec 28: ... until github has a syntax for linking to a function definition I think it's better than nothing.

These links are from text that describes one named function calling another named function. There is no guessing involved, if the link does not land on the right line, the referenced function definition will be easy to find in that file.

While I understand what you're saying about permalinks, I made a conscious choice to use links that are approximate, but I believe, better than nothing. The reality of URLs is that they go stale over time. Readers have encountered this before and have skills for dealing with it.

The way I see it...

Future Case 1: Links remain close enough that the reader can easily follow along in the code as they read the doc.
Action: do nothing.

Future Case 2: The links end up woefully wrong (Note: if things have changed a lot, the probably the call path that is described in the doc is also out of date).
Action: Delete the doc -- "this doc is too out of date to be useful"; or update the links --- "someone changed the code but forgot to update the docs"; or reduce the links to just pointing to the files -- "keeping these line-number links up to date turns out to be a pain, lets just link to the file".

I'm offering up these notes as-is in case they are useful. If they don't meet the standard required for doc/devdocs, maybe its better if I make a julia-dev post (that was my initial intention, but I thought they might be more easily discovered by the relevant audience in doc/devdocs).
Unfortunately, I am currently unable to spend more time on this.

timholy · 2015-02-21T17:25:22Z

Despite my concerns over the links, there's good material here. I'm inclined to merge, and then if some sections become too outdated to be useful, we can just delete them.

ViralBShah · 2015-02-21T17:36:26Z

Go for it.

devdocs notes on memory layout and runtime initialisation

timholy · 2015-02-21T20:41:58Z

Thanks again, @samoconnor!

vtjnash · 2015-02-21T23:43:55Z

doc/devdocs/object.rst

@@ -0,0 +1,88 @@
+******************************


@vtjnash reminder to self: correct this section in #7906

tkelman · 2015-08-06T06:01:04Z

The line numbers in the links here are now failing linkcheck because much of the content of this file was reorganized. So we can either update all the line numbers or remove them.

tkelman · 2015-09-03T08:28:54Z

If no one wants to fix the line numbers in init.rst I'll be removing the code links.

samoconnor added 2 commits December 27, 2014 16:08

notes on object layout

1b8dcf1

Notes on runtime init.

af7bd4d

fix typos

2ec44e9

Keno reviewed Dec 27, 2014
View reviewed changes

Upadates per @Keno's feedback.

78474c4

vtjnash reviewed Dec 28, 2014
View reviewed changes

samoconnor added 3 commits December 28, 2014 18:30

tweaks per feedback

25006aa

64-bit note

c7e4ff7

note about 8-byte "nothing" object

8b88593

vtjnash reviewed Dec 29, 2014
View reviewed changes

Clarificaiton of nothing and singletons

7acbfc6

samoconnor mentioned this pull request Jan 3, 2015

Parse all command line options in repl.c #9482

Merged

timholy added a commit that referenced this pull request Feb 21, 2015

Merge pull request #9468 from samoconnor/doc_notes_branch

65ddb08

devdocs notes on memory layout and runtime initialisation

timholy merged commit 65ddb08 into JuliaLang:master Feb 21, 2015

vtjnash reviewed Feb 21, 2015
View reviewed changes

doc/devdocs/object.rst

@@ -0,0 +1,88 @@

******************************

Copy link

Member

vtjnash Feb 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vtjnash reminder to self: correct this section in #7906

tkelman mentioned this pull request Aug 11, 2015

0.4 roadmap #11536

Closed

13 tasks

tkelman mentioned this pull request Jan 21, 2016

Consistency of read() and write() functions (per #14608) #14660

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

devdocs notes on memory layout and runtime initialisation #9468

devdocs notes on memory layout and runtime initialisation #9468

samoconnor commented Dec 27, 2014

tkelman commented Dec 27, 2014

timholy commented Dec 27, 2014

Keno Dec 27, 2014

vtjnash Dec 28, 2014

samoconnor Dec 28, 2014

vtjnash Dec 28, 2014

samoconnor Dec 28, 2014

vtjnash Dec 29, 2014

samoconnor commented Feb 2, 2015

timholy commented Feb 2, 2015

samoconnor commented Feb 2, 2015

timholy commented Feb 2, 2015

samoconnor commented Feb 2, 2015

timholy commented Feb 21, 2015

ViralBShah commented Feb 21, 2015

timholy commented Feb 21, 2015

vtjnash Feb 21, 2015

tkelman commented Aug 6, 2015

tkelman commented Sep 3, 2015


		.. sidebar:: `special case. <https://github.com/JuliaLang/julia/blob/master/src/jltypes.c#L2973>`_

		The special singleton object :data:`nothing` is "void".

devdocs notes on memory layout and runtime initialisation #9468

devdocs notes on memory layout and runtime initialisation #9468

Conversation

samoconnor commented Dec 27, 2014

tkelman commented Dec 27, 2014

timholy commented Dec 27, 2014

Keno Dec 27, 2014

Choose a reason for hiding this comment

vtjnash Dec 28, 2014

Choose a reason for hiding this comment

samoconnor Dec 28, 2014

Choose a reason for hiding this comment

vtjnash Dec 28, 2014

Choose a reason for hiding this comment

samoconnor Dec 28, 2014

Choose a reason for hiding this comment

vtjnash Dec 29, 2014

Choose a reason for hiding this comment

samoconnor commented Feb 2, 2015

timholy commented Feb 2, 2015

samoconnor commented Feb 2, 2015

timholy commented Feb 2, 2015

samoconnor commented Feb 2, 2015

timholy commented Feb 21, 2015

ViralBShah commented Feb 21, 2015

timholy commented Feb 21, 2015

vtjnash Feb 21, 2015

Choose a reason for hiding this comment

tkelman commented Aug 6, 2015

tkelman commented Sep 3, 2015