Problem with linux repo for 3.10.18+ commits

[MrEngman]

I've been compiling modules with older versions of raspbian, downloading the kernel source from the git linux repository. I originally loaded the source using "git clone" and I select the particular kernel I need to compile for using "git checkout commmit-ID".

I have, however, found a problem. When attempting to load the source for any of the several kernel versions available for 3.10.18+ I am getting an error, for example:-

git checkout 49e62a27664ac2aaa3e85ba469e8b6857a242264
fatal: reference is not a tree: 49e62a27664ac2aaa3e85ba469e8b6857a242264

Opening https://github.com/raspberrypi/firmware/commits/master and browsing the code for commit ID 35d9905adcde341c82e899ae619882ffec836b6a for Nov 15 2013 ./extra/git_hash shows the linux source commit ID I tried to use to checkout the linux version I wanted but checking the linux commits it does not exist. This appears to be the same for all versions of the source for 3.10.18+.

Using sudo rpi-update commit_ID as listed below I can set my SD card image to use any of the 3.10.18+ versions of raspbian linux shown. If I then look at the Raspberry Pi firmware repo the commit-ID's coincide with those from https://github.com/Hexxeh/rpi-firmware/commits/master but the commit ID's for the kernel sources in ./extra/git_hash do not exist in the Linux source repo.

For the current rpi-3.10.y tree commits for other kernel revisions after 3.10.18+ appear
to be OK.

As an alternative to using git and git checkout I tried downloading the archived files for
the commits in question. They are all available in the archive and I've downloaded the zip files but I'm having difficulty extracting the files. I've tried both Windows, using winzip and winrar, and unzip on linux on the Pi but get wierd errors on both systems while unzipping them. I've tried downloading the archives several times and find no difference between them so I don't think there is a problem with the zip files as such. Do I need to do anything special to unzip them or is there an alternative approach to downloading the linux source.

The commit id's for https://github.com/Hexxeh/rpi-firmware and equivalent commits for
https://github.com/raspberrypi/firmware with the list of ./extra/git_hash values and thus
linux source commit id's is below.

Hexxeh rpi-firmware commit history - master
-------------------------------------------
rpi-update firmware Commit ID
https://github.com/Hexxeh/rpi-firmware/commits/master
-----------------------------------------------------
Nov 15, 2013 - 16d5a0c712e00256f96dff6c0d3caf1499dfe2bf - #596 3.10.18+
Nov 13, 2013 - 210a5c90340dd96c85fa3c9c11d6be032f553eba - #594 3.10.18+
Nov 12, 2013 - e40d8b4f5b06a19ebcb3dc1ab806339e01ee319f - #592 3.10.18+
Nov 11, 2013 - 4b1b61b328d1b8b1a0b7eaa48ca99a3f73677fac - #590 3.10.18+
Nov 10, 2013 - 8fd111f77895450323abc5b34efde19548ffc480 - #587 3.10.18+
Nov 07, 2013 - 964576cffdb8097ce65ce36ae07c25d951226cb7 - #585 3.10.18+
Nov 07, 2013 - 5f4e0ba5cf948a8f03c040602d17b57709a49c7d - #585 3.10.18+
Nov 06, 2013 - 4dd2c5de608ec1756dc6db5a423eaa076b32f618 - #579 3.10.18+
Nov 05, 2013 - dc709fae6f7fca6d1062dd49ef3527b27439ca73 - #577 3.10.18+



History for Raspberry Pi firmware / extra / git_hash
----------------------------------------------------
           Raspberrypi/firmware Commit ID
https://github.com/raspberrypi/firmware/commits/master
------------------------------------------------------
Nov 15, 2013 - 35d9905adcde341c82e899ae619882ffec836b6a - #596 3.10.18+
Nov 13, 2013 - 1100e2a7c36be9cab48f4f3a8b1dd340d8729ad1 - #594 3.10.18+
Nov 12, 2013 - 3e92add424c3c2617d5df65caa307c808017a9ea - #592 3.10.18+
Nov 11, 2013 - 1df86f8d19e7303e0c8bdd3936c3687c3d400a92 - #590 3.10.18+
Nov 10, 2013 - 18a163f9341755b00841312af2878afeb64c131c - #587 3.10.18+
Nov 07, 2013 - e85b566308a06952103fc3c70f46f3d13aac557b - #585 3.10.18+
Nov 07, 2013 - 452637c34ceb3f3323e09b56c705f34bbe3a4594 - #585 3.10.18+
Nov 06, 2013 - 5155bb371389439d828c39bb50eafa47ea5a1860 - #579 3.10.18+
Nov 05, 2013 - 5343d029a5f2f09bd9801914eac0d4bf95cadc1f - #577 3.10.18+

https://github.com/raspberrypi/firmware/blob/Commit-ID/extra/git_hash from
browsing commits above.
Points to Raspberrypi/linux source Commit ID
https://github.com/raspberrypi/linux/commits/
----------------------------------
Nov 15, 2013 - 49e62a27664ac2aaa3e85ba469e8b6857a242264 - #596 3.10.18+
Nov 13, 2013 - 2e9628b0324f34f3251f9c1858d403f4502bf84a - #594 3.10.18+
Nov 12, 2013 - ae937f99fee8a37f2ddd7270f6bcc0e497e8c903 - #592 3.10.18+
Nov 11, 2013 - f7c53d79dc2de683cf37ee36b1068c0ccff1e5c0 - #590 3.10.18+
Nov 10, 2013 - 7d307d7ecc23682a5c80f6487e665d56d0e40ae1 - #587 3.10.18+
Nov 07, 2013 - ce6ae7c6f9e323e646758473d98d2e12ec82607e - #585 3.10.18+
Nov 07, 2013 - ce6ae7c6f9e323e646758473d98d2e12ec82607e - #585 3.10.18+
Nov 06, 2013 - a987a783807aeed2232280248a49ed8fa5981858 - #579 3.10.18+
Nov 05, 2013 - 5836aff020cae396c577ff27d7a090d19f8e86b0 - #577 3.10.18+

[popcornmix]

My git tree seems to see the commit you referenced okay:

dc4@dc4-arm-01:~/raspberry/arm_linux/linux$ git show 49e62a27664ac2aaa3e85ba469e8b6857a242264
commit 49e62a27664ac2aaa3e85ba469e8b6857a242264
Author: popcornmix <popcornmix@gmail.com>
Date:   Fri Nov 15 12:38:16 2013 +0000

    config: Set CONFIG_SPI_SPIDEV=y

diff --git a/arch/arm/configs/bcmrpi_defconfig b/arch/arm/configs/bcmrpi_defconfig
index 0b3177b..20c1c4f 100644
--- a/arch/arm/configs/bcmrpi_defconfig
+++ b/arch/arm/configs/bcmrpi_defconfig
@@ -531,7 +531,7 @@ CONFIG_I2C_CHARDEV=m
 CONFIG_I2C_BCM2708=m
 CONFIG_SPI=y
 CONFIG_SPI_BCM2708=m
-CONFIG_SPI_SPIDEV=m
+CONFIG_SPI_SPIDEV=y
 CONFIG_GPIO_SYSFS=y
 CONFIG_W1=m
 CONFIG_W1_MASTER_DS2490=m
dc4@dc4-arm-01:~/raspberry/arm_linux/linux$ git checkout 49e62a27664ac2aaa3e85ba469e8b6857a242264
Checking out files: 100% (1214/1214), done.
Note: checking out '49e62a27664ac2aaa3e85ba469e8b6857a242264'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 49e62a2... config: Set CONFIG_SPI_SPIDEV=y

You don't want to download the zip file for the linux tree - there are long filenames and shortcuts it can't handle. Instead of downloading the zip file, replace zip with tar.gz in the URL and you'll have a lot more luck.

[MrEngman]

Hi popcornmix,

That's just what I'd expect to see regarding detached head etc., or at least very similar, however, if I look here https://github.com/raspberrypi/linux/commits/ and search back through the many pages to the approximate dates of the commits I cannot find the commits in question.

Anyway I'll try your suggestion to use tar.gz with the archives and see what happens.

Thanks
MrEngman

[popcornmix]

You can view a commit on github - no need to search:
https://github.com/raspberrypi/linux/commits/49e62a27664ac2aaa3e85ba469e8b6857a242264

[MrEngman]

That's what I want but trying to download it is a problem.

Thanks for that pointer. Looks like I can view the other commits I was looking at as well using the same format link. Strange they're not visible in branch: rpi-3.10.y. I'm certain they were when they first appeared last November, although I do seem to remember the commits list changed a short time later as I check for updates frequently, at least daily, to make sure the wifi driver module I compile and make available for other users to download is available for the latest raspbian update whenever needed.

I tried your suggestion for using tar.gz instead of zip and I get an error " ERROR 406: Not Acceptable." on both the Pi and my Windows laptop. Not sure why that is happening.

OK, think I might be getting somewhere. I checked the zip download link when downloading a zip file and it was https://codeload.github.com/raspberrypi/linux/zip/49e62a27664ac2aaa3e85ba469e8b6857a242264 so I tried changing zip in the link ..ux/zip/49.. and replacing it with .../tar.gz/.. I ended up managing to download a tar.gz file.

Just need to see what happens when I extract it. Ok, extracted it to my Pi and I've been able to configure it to enable me to compile my modules - make mrproper, make bcmrpi_defconfig, make modules_prepare, download Module.symvers. And now its in the process of compiling the module so all is looking great, :)

Thanks very much for your help. Really appreciated.

MrEngman

[MrEngman]

Seems like I'm learning quite a few useful things trying to solve this problem of mine.

The module compile failed and I found the Module.symvers file I downloaded ended up as a html file. I viewed the raw image for Module.symvers in my web browser and did a copy and paste to get a valid version. Took a while to figure out the right form of URL to download it with and then I ended up with the TLS issue when trying to download with wget like with rpi-update recently. Figured out how to download using curl by looking at the rpi-update code.

I have now been able to compile my module and it is working with a 3.10.18+ image on a Pi A.

Just one final issue. The vermagic format of the new module is rather odd and I had expected it not load and to give me a error saying vermagic had a problem. I've seen errors with vermagic before stopping modules loading but it didn't error and loaded OK so that's a little confusing.

Previous modules I've compiled show vermagic with a plus sign after the kernel version number. eg 3.10.18+, however with this new module there is just the kernel version number 3.10.18 and no plus sign. Any idea why it's like this? Would be interesting to know.

Anyway, Thanks very much for your help. I've learned a lot which is useful and appear to have got round the problem I was having so probably time to close the issue.

[popcornmix]

I believe the + means it was built from a git tree that wasn't clean:
http://stackoverflow.com/questions/19333918/dont-add-to-linux-kernel-version

I guess that is due to the build scripts on the build machine used.

It would probably be preferable to add "LOCALVERSION=" (or commit/move the build scripts) on our build machine so the "plus" disappears.
I'll try and do that when we move to 3.12 kernel tree (should be soon).

I don't believe this will cause an incompatibility with modules.