-
Notifications
You must be signed in to change notification settings - Fork 1
Space Apps Weekend
Work in Progress
During the event I was asked if I could write up my experience of the weekend, I agreed but it occurred to me after that I don't actually have anywhere to put a write of the weekend. As this is a write up of a code writing event I thought what better place to write it up than on the wiki for my code itself.
I have a few friends that have been to a hack camps before, but I myself had never been to one and didn't really know what to expect. My friend Grace (who helped me out with this code) told me about the event to which I signed up for straight away.
Before the day I had looked though the list of challenges several times and had found a few that I would be interested in. It wasn't until the day before the event when thought I'd have one more look that I noticed the Lunar challenge, this immediately caught my attention and I knew it would be the one I would end up doing.
On the morning of the first day you couldn't have pictured a more stereotypical room of geeks (sorry I mean technical people) better, we were all standing round with phones and drinks in silence. Taking the dive into the social unknown I decided to ask the room in general what challenges people were interested in doing.
During the next hour of eating biscuits drinking tea and talking with other attendees i'd met quite a few interesting people and had some rather surreal conversions. This led to the introduction to the weekend on ISIC's 28 monitor screen.
There weren't as many people at ISIC as hoped but there was enough of us to split into groups and do what we wanted. Sadly Grace was unable to find anyone that wanted to do the same challenge as she did so she offered to help me with the lunar app.
So armed with white board sheets and dry wipe pens I went to town with drawings and discussion of how the app should work. My idea:
- Read in the image and convert to altitude (If only it was that simple)
- Pre-process the data into sets (20km, 30km, 100km squares)
- Display full moon maps
- Allow high resolution analysis on small areas
Our aim was to be able to get a full moon map by the end of the weekend.
The first challenge we faced was that fact we only had an image to work with, I originally expected to be able to get hold of a data set of just lat/lon/altitude, but faced with a rather large image (~16 million pixels) I set out to learn how to read images in Java.
A couple of hours later I knew how to do it but Java couldn't read the image, some furious googling later it turns out that Java's Advanced Image API can't read TIF's compressed using Adobe, which just happens to be what it was compressed with. Next challenge decompressing the Tif. I have to hand it to Grace here, she kept me going on this, I was getting close to giving up hitting wall after wall with this code is somewhat de-moralising.
With an uncompressed Tif in my hands and having just ate I could start running the code. One issue with trying to read in an entire image that is ~47mb is you tend to cause Out of Memory Exceptions. Apparently creating a map with 16 Million entries is a little extreme for Java.
So, read each row into Java, write each row to file, read each row in, convert RGB to lat/lon & Altitude, write to file, read each processed row in, write to DB. How slow can that be? (I'll come back to this)
Grace had written a class that would take the colour scales palette and read each pixel in a column giving it an RGB value to an altitude. The next challenge, matching the colours. As I know now, the palette only created 389 possible options in an image with 255^3 possible colour combinations (16581375). By this time it was getting late, our progress had been slow, our original idea was by today to have a database full of data that we could query and general analysed data with, but all we have succeed in doing was getting to the step before where I expected to have started.
At about midnight I had very little brain power left and was trying to work out a efficient and working way to match the RGB values from the image to our palette, after several re-writes of the code I finally got it working (at about 3am). Thinking to myself it shouldn't take too long to read the image in, convert RGB to Altitude and then write it to the database I packed up and heading for bed.
Up at 8am, coding by 9am.
I kicked off my code and went to get a Tea. You know when I mentioned about how slow could it be? lets just say very very slow. It would take about 4-6 hours to get the image processed and into the database, which wouldn't of been too bad if it weren't for lots of little bugs having crept into my code.
So while the processing attempts to complete I began working on the analysis class, that would take an area of x km, and work out roughness based of slope angles between each pixel then giving the entire square an average roughness and altitude.
With time rapidly running out it started to become clear that we weren't going to have anything to show for a presentation.
It was also this day that I got a chance to talk with Nick about what it is that we were trying to do and what we thought of the event. I loved the event (despite the stress and problems we'd faced) and I talked him through what we were trying to do and what the problems we faced were, it was then that he said he'd probably be able to get me the data that I would of wanted.. if it wasn't a weekend.
So with very little to do but watch text files accumulate on my hard drive I had another Tea and explored the ISIC a bit more.
Presentation time rolled around and I only had a partly filled database and nothing to show. So with some enthusiasm I got up in front of everyone and talked about what we had done, what problems we faced and what we intended to do. Ok so it wasn't as impressive as some of the other presentations, but I have to hand it to them, some of the stuff they produces was incredible (two of which won not only locally but in global judging).
So after the presentation I had another chat with Nick about continuing development of the code and trying to get hold of some data sets.
It's been about 7 weeks since the weekend I have done quite a bit of coding work on the application, the code can create KMZ's for the moon but the longitude was off by 180 degrees so a few tweaks to make.
I have contacted Nick and with some other contacts trying to see what data is available for me to use in this application.
There is still much development needed and some serious improvements for speed but it's definitely (albeit slowing) getting there