Sprints

Sprints

Sprint 2004.03 - Berlin

Stackless sprint held in Berlin from 10th-14th March 2004.

Christian Tismer

So here is what I did:

:code-block:`Image:christian_tismer.jpg,nolink`

On the first day I gave a talk which should introduce everybody to Stackless and how it works. With bad preparation, I failed to do a good job on motivating beginners, and Martina dropped off, disappointed. (I still hope to get her interested in PyPy.) Later on, I could make it a little better, not always diving deeply into internals but showing the current API and the thoughts behind that. On the second day, I gave a couple of interactive debugger sessions, showing how the interpreter normally works, how it works in Stackless mode, and how switching between tasklets happens. I think this was quite informative for everybody. Lutz took video recordings, and I'm curious to see them on this site! We had many discussions about the API, and the idea came up that tasklets are even a too large design with too much predefined behavior, and it would be better to find a smaller structure and to implement tasklets on top of it. Everybody was trying to make some sense out of Stackless, to learn how to use it, and I gave as much help as I could. Some concepts were hard to explain in newbie recipes, and we added a few new methods which look more natural to beginners. I did some bug fixes and small changes to support the new pre-emptive scheduling, and although I wanted to avoid deep hacking by myself, I wasted almost two days trying to remove the f_callee field and replacing it by a better concept, but failed. Enhanced pre-emptive scheduling a little by making sure that a tasklet that was in atomic mode while an interrupt occured gets the delayed interrupt request immediately after it released the atomic flag. One of the seldom changes which work instantly. Armin asked me many questions about the hows and whys of the implementation, and he asked me astonishingly much about the hardware switching, intial stub, initialization and all the complexity which is introduced by the need to have a main tasklet in place. On the last day, he presented me a very small, very clean implementation of "greenlets", which do the hardware switching stuff with the existing library, but implemented as an extension to regular Python! Armin and I had a very long discussion last night, and we found out how to harmonize soft and hard switching in a single concept, which is applicable to any programming language. This is the begining of a completely new implementation of the Stackless kernel, and most probably of a new, general green thread package which is as efficient, clean and small as nothing else before. Things I'm going to do: Continue to take care of Stackless of course, making sure that all the new developers stay happy and that they continue to provide more such great documentation, tests, examples, and enhancements. Write documentation, sample code, explain the inner machinery, as it is now and as it should be in the future. Enhance the functionality of the website, to make it easy to upload sample code which should possibly also run on the site, have downloadable installation packages, allow users to add to wiki pages, and more. The following is a list of things to newly develop, but I don't mean to do that alone. Participation is most welcome, this is more a to-do list than my own commitment: Re-implement microthreads based upon tasklets, since people are asking for this since 2002. Create a new branch for Stackless for Python 2.4, in order to be ready with a Stackless release when Python 2.4 is ready. Create a new branch of Stackless which is the bare minimum to support non-recursive calls, no tasklets, no pickling, minimizing the patches to the Python code base. Create an extension module which does the pickling "from outside", without patching the built-in types in any way. Instead, everything will register itself in the copy_reg module. Create new hard and soft switching based upon the phantastic "greenlet" idea. I'm undecided whether to do it as an extension in the first place, but the chance to get rid of most of the current machinery is most challenging. Create tasklets and channels as a Python module, based upon greenlets, for testing and documentation purposes. Implement tasklets as a C extension, based upon greenlets, for compatibility purposes. All I forgot to mention here should be added, virtually. There are lots of other things to do, especially supporting disemination. My heavy side was always on technical issues, but fortunately I'm no longer alone and can rely on the help and support of the group, which makes me very happy. Sincerely -- chris

Dr. Armin Rigo

So here is what I did:

:code-block:`Image:armin_rigo.jpg,nolink`

I did not contribute a lot of code, but instead spent the whole week trying to understand how Stackless actually works with the help of Christian. I extracted from the Stackless source a small C extension module for regular Python that implements hard-switching, which exposes a very basic low-level interface on top of which arbitrary things can be implemented by the user (like tasklets, or generalized generators). This is what I'm planning to do: Continuing to lobby Christian for a simpler interface and implementation of Stackless :-)

Bob Ippolito

What I've done this week in Stackless CVS:

:code-block:`Image:bob_ippolito.jpg,nolink`

Brought in Python 2.4's copy module, because the one we had just didn't work correctly. There are slight modifications to remove support of set/frozenset, as those builtins didn't exist in 2.3. Workaround for a GCC bug that prevented the stack switching code from compiling because it would forget to include the full PIC prologue. This only happened with the most recent developer seed of Xcode 1.1.1 - gcc (GCC) 3.3 20030304 (Apple Computer, Inc. build 1636) (-mlongcall). Added experimental soft switching for channels and tasklet exceptions, however, they are #define'd out at the moment because they are not stable. Various bugfixes. And I plan to, sometime in the near future, when I'm not working on my PyCon talk: Finish channel soft switching. Help develop the scheduler-less lighter-than-tasklets-and-channels objects that were discussed, such that we can make the core smaller and more generic. Package up an easily installable framework build of Stackless for OS X when it's stable enough to do so. Work on networking code that uses Stackless, and Mac-specific examples.

Stephan Diehl

Here is what I did:

:code-block:`Image:stephan_diehl.jpg,nolink`

I started on a stacklessness implementation. Basically, if it's done, tasklets will be simulated by real threads. When it's done, it will probably only support a subset of the current stackless implementation. For example, there won't be (and never will be) any pickling support. And what I'm going to do: I'll continue on it later this week. The whole idea of stacklessness might be no longer needed if Armins extension module (hardswitching) is working. But it will be still very educational.

Dinu Gherman

Ok, here's what I did:

:code-block:`Image:dinu_gherman.jpg,nolink`

Started a "Stackless Glossary" Zwiki on stackless.com Wrote a script to convert it into a compact download-/emailable PDF (needs LaTeX and Docutils, so I'm not sure where to put it - Chris?) Started building a "Stackless Tutorial" thingy with lots of sample code (together with Axel, needs more work) Added a (simple) "umbrella" unittest script to run all unit tests (Stackless/unittests/runAll.py) Suggested API changes like the call-like argument setting thing on tasklets Dito for stackless.getmain()/getcurrent() Started converting some tests to unittests and adding more (e.g. Stackless/unittests/test_chan_cb.py) Asked many questions of varying stupidity Teased my brain with quite a few answers provided by Christian And here's what I'm planning to do: Tell people to jump on extending the "Stackless Glossary" Continue with Axel on a code-rich "Stackless Tutorial" Write something to visualize communication between tasklets Write an article, codename "10 things I learned about Stackless" (feel invited to provide any such items, please!) Add more unittests That's it for today and thanks for the fish!

Axel Bringenberg	... and here is my little summary:
:code-block:`Image:axel_bringenberg.jpg,nolink`	I joined the sprint as a stackless newbie dummy, Tried to figure out the way (with all these mines and pitfalls) of a normal stackless beginner. In this role I created a lot of ten-lines-examples, the base of a didactic step-by-step beginners tutorial. And in future? Well, there's a lack of documentation which shouldn't be ignored Thanks for all, bringi

Tomek Meka	This is what I did:
:code-block:`Image:tomek.jpg,nolink`	I've joined the sprint on Friday, so I missed the Christian's introduction and I spent my time with building the new stackless site in plone and convincing the sprinters, that plone is not that bad :-) This is what I'm planning to do: I'm going to see the recorded introduction to stackless and to write some examples.

Lutz Paelike

What i did:

:code-block:`Image:lutz_paelike.jpg,nolink`

Filming the presentations and discussions to make them available for people who did not manage to come to the event. Starting to port the OpenSteer Library to (stackless) python When it's finished it should be an authoring environment/testbed for creating virtual agents This will finally be a more complex demo application what you can do with stackless. It's still work in progress and you can't use the sprint results for something useful right now but stay tuned Outlining together with Stephan Diehl an authoring tool for tasklets and channels. This will basically be a node based authoring tool and you can connect nodes (tasklets) with wires(channels) to move your data around. Lot of inspiration comes from the commercial graphics applications Houdini and Shake This can maybe end up in a plugin for Boa Constructor. Helping Tomek a little bit to set up the web site What i will do: Melt down all the hours of video material to easy consumable bits, compress it for internet distribution and publish it on this site. This will take a while i'm afraid because lots of data has to be processed. After completing the port of the OpenSteer Library integrate stackless specific structure Construct higher level applications based on stackless, mainly 3d graphics related Integrate Stackless as the Basis for my Python Plugin for Maya instead of regular Python and release it

Stefan Drees

My sprint-summary:

:code-block:`Image:stefan_drees.jpg,nolink`

Always a great time, if someone opens up and a team is willing to close on in (attention sloppy language usage;) or as some of our kids say _reeeespect_ chris. Count me in for the next _Zwischensprint_ in Berlin! Came late friday evening, had to perform some extra steps to sync with the team's current tasks Came closer to the point with stackless, so to be able in helping in not letting the core ideas become pointless. Began to wade through the C-code. Performed clean-up and patching tasks where needed. Discussed test and validation strategies with Dinu and Chris. Patched some python core tests (generator, descr) that tested for some silly conditions, failing with stackless. My future plans: Enhance funding for stackless. Hm, can you do this too ;?). Help: hacking the C-part of it help with the greenlets keeping the linux2 plattform working enhance testing (function and performance) Contribute to Wiki and Cookbook on stackless.com Stay tuned All the best, Stefan.

Richard Emslie

Here is a brief summary of my sprint checkins:

:code-block:`Image:richard_emslie.jpg,nolink`

Added unittest to check life cycle of tasklet during watchdog runs Added suite of unittests to check watchdog behaved under number of conditions. (Yes the above 2 should be merged - and documented better!) Hacked the watchdog to work with soft switching. Some work needs done to work better when pickling (a flag added to pickle state for example) Demo code to show preemptive autoscheduling with running watchdog functionality. On top of that I mainly tried to absorb an awful lot of information about CPython and stackless implementation - probably not too successfully, but happy that I know an awful more than before. What will I do now: Seems watchdog broke on final day - so will fix that. Integrate with pickle flag. Will maintain autoscheduling/watchdog, fix bugs etc if anyone is using this feature in the current version. Possible join in on an effort to implement uthreads library. Want to strip down to bare minimum stackless and implement global generators. I know Armin has does something similar already, but there is nothing like reimplementing the wheel! Keep abreast of "greenlets" (nice name :-)) and see if I can add any input there. A lot of good ideas were raised during the sprint and Christian got a little beaten up over things from interface to core implementation. I guess that's why we have sprints. However, I think everyone should bow lowly and appreciate all the hard work he has put into stackless, which we can now captalise on, and hopefully create great new things. Thank you Christian. And thanks to anyone who helped organise and make the sprint happen. Cheers, Richard

Sprint 2006.02 - Dallas

The Stackless sprint held in Dallas, Texas in the USA from the 27th February to the 2nd March 2006 at PyCon.