0.4 roadmap

[StefanKarpinski]

Done:

• refactor abstract array type (RFC: Give AbstractArrays smart and performant indexing behaviors for free #10525)
• change colon lowering (RFC: Move Colon translation out of the parser #10331)
• serializing with cycles (RFC: handle serializing objects with cycles using a Serializer object #10170)
• bare doc strings? (abbreviated syntax for docstrings #8965)

Remaining:

• package precompilation (WIP: static compile part 4 (user-interface) #8745)
• reserve field default syntax (default field values #10146)
• make [a,b] non-concatenating (syntax: separate array concatenation from array construction #7128)

Potential:

• reserve triangular dispatch syntax?
• convert default values?

No consensus reached:

• ordered dicts? (WIP: try ordered Dict representation #10116)
• zero initialization? (zero out memory of uninitialized fields #9147)

Deferred:

• defer method ambiguities to runtime?
• change constructor syntax? (cc @JeffBezanson)

[tkelman]

#10170 was already merged. I think #10331 is a part of #10525 but @mbauman can correct me if I'm wrong.

edit1: #11242 is worth seriously thinking about on this list

edit2: we'll also have to fix the non-reproducibility of git-externals before tagging, ref #10743 (comment) done

[mbauman]

I think #10331 is a part of #10525

Not quite - #10525 allows indexing with Colon(), but it doesn't change the parser lowering of [:]. That's what #10331 provides (there are two atomic commits there - once #10525 lands I'll discard the first).

[JeffBezanson]

I'm going to move all strictly-bug and -performance issues from the 0.4 milestone to 0.4.1.

[tkelman]

Better to keep the distinction clear between 0.4.x and 0.4.1, otherwise that's going to get really annoying when it's time to actually tag 0.4.1 as a bugfix release.

[JeffBezanson]

Yes, I think I meant 0.4.x.

[sbromberger]

I really hope we can keep #9147 in 0.4-release... this one just bit me again.

[jakebolewski]

Why is #9147 on the release roadmap? Of all the changes to make it does not really seem to be the most pressing one, and it can always be added in the future as a non-breaking change.

[JeffBezanson]

I'd really like to have it, but you're right that it's non-breaking and technically doesn't need to be release blocking.

[sbromberger]

It's not critical to the release but it's a security-significant change that fixes an issue that's caused large amounts of grief to various packages.

In the event we don't get this into the release (which would be a tremendous shame) would you be amenable to a documentation PR that states, effectively and emphatically, DO NOT TRUST UNINITIALIZED VARIABLES?

I should add that it's a relatively old issue as well, initially raised by none other than Ron Rivest himself (who might happen to know a bit about the security implications here :) ).

[carnaval]

I don't see why zero fixes it though. At least garbage is obvious most of the time (we could actually have a guaranteed garbage mode as a debug thing to spot usage of unintialized stuff). For example I use a small integer field as a flag for some things and zero has a very special meaning. I find it dangerous to assume that zero is always a safe choice.

[sbromberger]

@carnaval I really think we should keep the discussion of the merits of the approach in the actual issue thread itself so we don't get two distinct threads.

[JeffBezanson]

Yes, please see the discussion in that issue, where many such points have been covered.

[ScottPJones]

What about Unicode bug fixes? 😀

[quinnj]

@ScottPJones, bug fixes are great for bug fix releases :) It's mainly breaking, larger behavior changes, syntax changes, etc. that are important to get in before an official 0.4 release so that those features are available for the entire 0.4.x release cycle where pretty much only bug fixes/performance improvements are allowed.

[prcastro]

What does "bare doc strings" mean?

[StefanKarpinski]

"bare doc strings" means being able to use plain strings in void context to provide doc strings instead of needing to use @doc macros.

[ScottPJones]

@quinnj But if the code is already ready, and many people have already said it should go in? Bug fixes help move the language forward also, they make the language more reliable.

[tkelman]

This issue is about which major breaking issues are going to determine scope completion for moving from 0.4-dev to 0.4-pre (feature freeze). Non-breaking bug fixes can continue at any time, though we will be especially conservative about only committing fixes for release-blocker bugs as we get closer to tagging the release (after branching for release-0.4, for example).

[timholy]

@ScottPJones, it's likely that there will be weeks/months between feature freeze and release, and the enthusiasm for bugfixes is especially high during that period. Let's keep this issue focused on breaking changes.

[ScottPJones]

@timholy Since certain people have said that they think my bug fixes might be breaking, wouldn't that mean that those changes definitely should go in earlier, rather than later?

[ViralBShah]

Bare doc strings are essential for package help, and I really would like to see those happen as part of the 0.4 release.

[ViralBShah]

There are 20 items in all marked as 0.4.0. Should we trim that label to "must do" items from this issue, and mark any important bug fixes slated for 0.4.0 to 0.4.x? That way, there is a clear line of sight to the 0.4 release.

[ScottPJones]

👍 to bare doc strings

[prcastro]

👍 to bare doc strings here also

[tkelman]

Should we trim that label to "must do" items from this issue, and mark any important bug fixes slated for 0.4.0 to 0.4.x?

I think Jeff already did that to a large extent. Anything he left in 0.4.0 was probably for good reason?

[ViralBShah]

Given that @lindahua is busy, is SparseVectors realistically possible? Perhaps it fits well with the Arraymageddon release anyways.

[ViralBShah]

Also, I am not sure we can take vector transposes seriously in 0.4, and Arraymageddon is the right release for it anyways.

[ViralBShah]

Range indexing producing subarrays is another thing that should be in the Arraymageddon release.

[tkelman]

Probably true on all counts. I'd personally like to see SparseVectors sooner assuming they could be finished in time before feature freeze, but it's not worth holding up a release over.

[tkelman]

Yeah just need a path towards what to do on #12489 and the related doc fixes.

[MichaelHatherly]

#12496 would be good to have for the methods that can't be documented at their definition site, but with #12511 that is a relatively small amount now - just those in essentials.jl and some of the stuff in Core. A bikeshed on the actual syntax for that would be worth having, maybe.

[MichaelHatherly]

There's also the "bindings" part of #12088 that would avoid documenting constants by their value rather than name. @one-more-minute would that be simple to split out?

[tkelman]

Oh, we also still need to do something about #9079 (comment), freezing due to files with the same names as packages isn't an acceptable thing to do to users. May or may not need to block RC1 for that, but it definitely needs fixing before final.

[ivarne]

I think release candidates sounds like a finished product without known blocking issues. When 0.3 was released, we used the -dev mark and planed to switch to -pre to mark feature freeze (which seems to be about now). The switch would be a signal to package authors that no new breaking changes where pending, so it is time to go through and make sure packages are updated.

alpha and beta releases would also be possible, but as our Version type orders alphabetically we'll have alpha < beta < dev which isn't as nice as dev < pre < rc. We also don't have the distinction between internal and external test versions, so I don't think the alpha/beta distinction makes sense.

I'd like to see 0.4-pre1 and 0.4-pre2 releases, if we want to release versions with known issues.

[ViralBShah]

I like the idea of pre releases. We could announce a feature freeze shortly and start with pre releases, switching to RCs then. This will also shorten the RC cycle.

[ViralBShah]

I also like the idea of getting a pre-release out right away, so that package authors can start preparing for 0.4 and start plugging into the new doc system.

[ViralBShah]

I also wonder if someone should be appointed as a release manager for this release, to make sure all the things happen. @tkelman is the natural choice, but he may not want further distractions in the last phase of his PhD filing.

[MikeInnes]

@MichaelHatherly sure, should be simple enough to pull out the bindings commit and redo the macro. I can't compile Julia easily right now so if you're interested go ahead. Would also be good to remove the objdoc fallback before an RC.

[ScottPJones]

RC should really be just that, a real candidate for release, so no known bugs (at least, bugs that people intend to have fixed upon release), so definitely it should be -pre, after feature freeze and branch off of master.

[MichaelHatherly]

@one-more-minute I'll have a look at it. Have you had a chance to take a look at #12511? Just rebasing it now.

[MikeInnes]

Ah hadn't seen that, will take a look ASAP

[tkelman]

How different would a pre-release be than our usual nightlies? I think now would be a good time to mark a feature freeze by changing VERSION from 0.4.0-dev to 0.4.0-pre though.

If someone wants to help with a simple but slightly tedious issue, the line numbers in doc/devdocs/init.rst are all wrong right now which causes make -C doc linkcheck to fail, ref #9468 (comment).

[IainNZ]

Pre-release in my mind is feature freeze, and is basically an open invitation for the "engaged" community to dive in (e.g. package devs). The RC is the risk-tolerant people, wider set. Not sure if I'm just making things up.

[ScottPJones]

@IainNZ "Release candidate", at least in my experience, has a very specific meaning, it should be something that, barring new bugs found in testing, will become the next release.
Whether you call the feature freeze version an alpha, beta, or pre-release version, that's a very different beastie from a real RC.

[ivarne]

How different would a pre-release be than our usual nightlies?

I would guess that a prerelease has a git tag and an announcement, which the nightlies doesn't have. The important part is that we begin to encourage users to check out 0.4, as we will stop pushing breaking changes that require users to change their code (unless they depend upon buggy behavior)

[ViralBShah]

Is #12435 also on the critical path here?

[tkelman]

Ref #12489 (comment). IMO no.

[tkelman]

Anything other than #12742 (done) left before branch and freeze?

(many doc fixes still necessary before rc or final though)

[timholy]

Would be great if @JeffBezanson would take a peek at #12544 and decide if it makes 0.4 or not.

But mostly I'm excited about getting it out the door!

[JeffBezanson]

Hopefully improving line numbers is not considered a breaking change, so not urgent for 0.4.0.

[timholy]

Sounds good to me.

[mlubin]

Given the feature freeze, time to bump to 0.4.0-pre?

[jskDr]

I am not sure whether the list is ordered by importance. However, I want to
say that ordered dict is highly important. As it is not supported Python
without using external library, dict() is convenient but complicated most
times.

James SJ Kim
sungjinkim@fas.harvard.edu

On Thu, Aug 27, 2015 at 8:20 PM, Miles Lubin notifications@github.com
wrote:

Given the feature freeze
https://groups.google.com/forum/#!topic/julia-dev/M6XERBx96ts, time to
bump to 0.4.0-pre?

—
Reply to this email directly or view it on GitHub
#11536 (comment).

Best regards,
(James) Sungjin Kim, Ph.D.

• Post-doc, CCB department in Harvard
  sungjinkim@fas.harvard.edu
  (Tech-consultant in Samsung Elec.)

[tkelman]

The list in the top post is meaningless at this point, features for 0.4 are frozen. Should probably close this issue since the scope has been determined, open a new one if necessary explicitly for tracking RC blockers.

[kmsquire]

@jskDr, I don't quite understand your comment about python, but there is an OrderedDict implementation in DataStructures.jl.

[jskDr]

Oh, really. I didn't know that. I will use OrderedDict whenever I need.
I thought it does not exist in Julia because ordered dicts (#10116
#10116) is listed above as a work
item.
Anyhow it is so appreciate as letting me know OrderedDict in
DataStructures.jl.

• https://github.com/JuliaLang/DataStructures.jl

James SJ Kim
sungjinkim@fas.harvard.edu

On Thu, Aug 27, 2015 at 10:45 PM, Kevin Squire notifications@github.com
wrote:

@jskDr https://github.com/jskDr, I don't quite understand your comment
about python, but there is an OrderedDict implementation in
DataStructures.jl.

—
Reply to this email directly or view it on GitHub
#11536 (comment).

Best regards,
(James) Sungjin Kim, Ph.D.

• Post-doc, CCB department in Harvard
  sungjinkim@fas.harvard.edu
  (Tech-consultant in Samsung Elec.)

[kmsquire]

(Mostly off-topic, sorry.)

The "ordered dict" item above was whether or not to replace the current Dict implementation in Base with an ordered dictionary implementation, which (if implemented correctly) can make traversal faster, but which has additional overhead in other ways. This has been decided against (because of the overhead), but it might be useful to replace the OrderedDict in DataStructures.jl with the one that Jeff created for testing.

Also, just to be clear, in Julia OrderedDicts maintain their insertion order. DataStructures.jl also has a SortedDict type, which maintains keys in sorted order.

(For anyone coming from C++, this might be confusing, as the default map class there is described as "ordered", meaning "sorted", and there is also an unordered_map.)