Roadmap to 1.0

[benjchristensen]

I want everyone to know where we're heading and what's left before we hit 1.0.

From the beginning we have allowed ourselves to do breaking changes on each 0.x release as we were aware that the design was not finished when we started this project. We are getting close to being done. Our goal is to release 1.0 in the coming months after stabilizing the API so that it can be relied upon without breaking every couple months.

Project Structure

When we hit 1.0 we intend on splitting out the language adaptors into their own top-level projects such as RxScala, RxClojure, RxGroovy, RxKotlin, RxJRuby etc.

This will allow each project to iterate as needed at their own pace, especially since some will need to continue iterating while the RxJava core stabilizes. For example, if RxScala needs breaking changes it can bump it's major version while RxJava does not. This is particularly important to RxScala for handling changes such as Scala 2.10 -> 2.11 -> 2.12 etc.

Major contrib modules will also be moved out, such as RxAndroid which also needs its own life-cycle.

Outstanding Work

The major items of work to be finished before 1.0 are:

• Backpressure: Backpressure #1000 [Completed]
• Serialization Behavior: Serialization Behavior #998 [Completed in new merge implementation]
• Scheduler API: Proposed Scheduler Interface Change for 0.18 (yes, again) #997 Completed
• Remove all deprecated methods/classes [Completed] #1053 #1621
• Finish migrating all operators to using lift and chaining the Subscription via Subscriber (current priority) [Completed]

The primary goal is to nail down the public API. New functionality can come in 1.1, 1.2, etc. The secondary goal is for all operators to work as advertised (regarding unsubscribe, back pressure and non-blocking). There will always be bugs, that's why 1.x.y will still be active after release, but the desire is to not need to ship 2.x soon after 1.x as this is a low level library that once entrenched becomes hard to migrate (we create significant pain at Netflix on each 0.x release).

Going Forward

Please comment if you feel there are other critical things to achieve before 1.0. The fastest way to getting us to 1.0 is helping us achieve the work stated above.

[benjchristensen]

Based on discussion with @headinthebox we will probably move rx.joins into a separate module prior to 1.0 since this API does not yet feel finalized. If over time it matures it can be brought back into core in 1.1, 1.2, etc ... but we can't leave things in the core that may need public API changes.

---

Completed in #1049

[benjchristensen]

Removing deprecated methods/classes. A few still remain to be done before 1.0 but can't be done until all operators are migrated:

• Observable.OnSubscribeFunc
• Observable.create(OnSubscribeFunc)
• SafeObservableSubscription
• SafeObserver

[benjchristensen]

Migrating operators to using lift and Subscriber is now the priority for making progress to 1.0.

I will personally be tackling the back pressure prototypes and eventually propose the change via a pull request. This work is being tracked at #1000.

[akarnokd]

I'd like to start working on these. Is the Scheduler API stable for now, so any changes to it won't require me to rebase? In addition, could you setup an issue with checkboxes dedicated to each remaining operator?

[benjchristensen]

Thanks @akarnokd, I really appreciate your help on this.

I believe the Scheduler changes are stable (unless someone quickly comes up with a better name for Inner). I'm just waiting on @mattpodwysocki to review the changes so we are in sync with RxJS/Rx.Net going forward before releasing 0.18.

On the topic of migrating all of the operators, I think there are a few things we should nail down as patterns we want to follow so operator implementations are similar (to ease ongoing maintenance).

Things I can think of are:

• a common, well tuned ring buffer (SPSC)
  • ring buffers will become useful in the async operators as well once back pressure allows us to be bounded
• how we apply locking/synchronization
  • we have various different approaches and styles right now
  • when should state machines be used?
  • when should synchronized be used?
  • when should lock be used?
• when should OnSubscribe vs Operator + lift be used?
• how should error handling be done?
  • see map as an example

Perhaps we should create a README inside the rx.operators package meant as an internal coding guide for decisions to these and other things we determine as practices we want to follow?

What patterns do you think we should be following to the above questions and others you are considering?

[benjchristensen]

Version 0.18.0 was released with the Scheduler changes, removal of rx.joins and majority of deprecation cleanup.

[akarnokd]

Sorry for the delay, but I needed to implement a few operators to establish a pattern for myself.

Ring buffer
Whenever buffering is required with fixed size, and there is only addition and removal, one should use ArrayDeque instead of LinkedList. Note however that ArrayDeque grows by doubling its capacity and may consume extra memory beyond the intended fixed size. Eventually, we might need a custom ringbuffer implementation.

Locking & Synchronization
For light contention, such as when an observable and a scheduled action races for a shared object, there isn't much difference between the two. Therefore, I prefer using synchronized as it is more readable. For heavy contention, state machines with randomized backoffs on CAS failures seem to be the way to go, but implementing such logic is not trivial. So in case a new concurrent operator needs to be implemented, start off by using synchronized blocks to establish the base behavior then let one of the experts rewrite it to a state machine.

OnSubscribe vs. Lift
Use OnSubscribe if the operator works as a primary source of events (from) or there are many source observables that need to be mangled together (merge). Use lift if the operator "modulates" a primary source of events (map).

[daschl]

@akarnokd wrt to backpressure and groing. Shouldn't - if you want to mirror a ringbuffer - the ArrayDequeue or whatever be initalized to the required size upfront so you dont need to grow it? You can preallocate the slots more or less, so you also have less garbage flying around (like with the LinkedList).

[akarnokd]

From the JDK source code ArrayDeque appears to round up the initial capacity to the next power-of-2 and doubles it everytime it grows. In addition, the usage pattern in RxJava is as follows: add, check-size, remove, i.e., if it is full, it will add the latest item, discover that it has overgrown the expected capacity and remove on item. If the capacity happens top be power-of-two, adding one element doubles the size which is then wasted.

[benjchristensen]

wrt to backpressure and groing

Once we implement #1000 there should be very few places left that require unbounded buffers and we can leverage ring buffers even in places such as observeOn and zip.