Upload new version to Debian

[fangfufu]

This new version contains a significant new feature - permanent cache system, and various bug fixes and enhancement. I think we should package this and upload it to Debian.

I know the man page needs to be updated.

Is there anything I could do?

[jcharaoui]

I'm going to look to push an update to Debian.

For now I just want to point out that you seem to have rebased the repository, which means that I now need to start with a fresh httpdirfs repository on my end. Please, in the future, avoid at all costs rebasing commits that have been pushed to the master branch, and instead use development/feature branches.

[fangfufu]

Yes, sorry, I did run the rebase command - I wasn't entirely sure what I was doing. I will bear that in mind in the future.

[fangfufu]

Ok, I added a new feature to cache the directory structure, hence the version number bump. Once you finish making all the changes, let's bump the version number again.

Once again, I am sorry for rebasing the repository - I definitely won't do it again!

[jcharaoui]

@fangfufu If I may make a suggestion: there's no need to create a new release every time you push a bugfix or feature. Instead, just push whatever you need to master. Once you're satisfied, let it settle for a while. Give users the chance to report bugs. Then when the code is stabilised and the last few commits are bug fixes, its a good time for a release. To be honest I don't feel it's very prudent to push code that has been just written and barely tested into Debian. This carries the risk of causing problems for users, which we want to avoid at all costs.

[fangfufu]

@jcharaoui , I agree. I don't have much software development experience, so I am still learning the convention and etiquette.

So I guess it might be a good idea to come back in two weeks and think about pushing the code to Debian. I think it is a good idea to let the code settle down a bit. I am currently testing the code by running it on my server. I am trying to cache a fairly large http directory.

However, I have to say that the code in the repository is way better than the version in Debian. In particular, release 1.0.3 fixes failure when encountering directory with long listing, and release [1.1.0] fixes a memory leak during LinkTable creation - I couldn't cherry pick bug fix part of the code, so I decided to release the new feature alongside with the bug fix.

[fangfufu]

@jcharaoui , Hey Debian Buster has been released. Shall we start working on this again?

[fangfufu]

@jcharaoui , hey is there anything I could do to help? :-)

[jcharaoui]

@fangfufu Could you update the changelog and tag 1.1.7 ?

[fangfufu]

I updated the change log and Makefile. I haven't created a tag for 1.1.7 yet, because you have made some changes, feel free to tag 1.1.7 once you finish making all the changes. :-)

[jcharaoui]

I tagged 1.1.7 but unfortunately it doesn't build with Debian unstable:

cc -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/httpdirfs-fuse-1.1.7=. -fstack-protector-strong -Wformat -Werror=format-security -g -O2 -Wall -Wextra -Wshadow -D_FILE_OFFS
ET_BITS=64 -DVERSION=\"1.1.7\" `pkg-config --cflags-only-I gumbo libcurl fuse` -Wl,-z,relro -Wl,-z,now -pthread -lgumbo -lcurl -lfuse -lcrypto `pkg-config --libs-only-L gumbo libcurl fuse` -o
 httpdirfs main.o network.o fuse_local.o link.o cache.o util.o                                                                                                                                 
/usr/bin/ld: network.o: in function `curl_multi_perform_once':                                                                                                                                 
./src/network.c:155: undefined reference to `curl_multi_perform'                                                                                                                               
/usr/bin/ld: ./src/network.c:157: undefined reference to `curl_multi_strerror'                                                                                                                 
/usr/bin/ld: ./src/network.c:176: undefined reference to `curl_multi_timeout'                                                                                                                  
/usr/bin/ld: ./src/network.c:188: undefined reference to `curl_multi_fdset'                                                                                                                    
/usr/bin/ld: ./src/network.c:191: undefined reference to `curl_multi_strerror'                                                                                                                 
/usr/bin/ld: ./src/network.c:206: undefined reference to `curl_multi_info_read'                                                                                                                
/usr/bin/ld: network.o: in function `curl_process_msgs':                                                                                                                                       
./src/network.c:102: undefined reference to `curl_easy_getinfo'                                                                                                                                
/usr/bin/ld: ./src/network.c:106: undefined reference to `curl_easy_getinfo'                                                                                                                   
/usr/bin/ld: ./src/network.c:110: undefined reference to `curl_easy_getinfo'                                                                                                                   
/usr/bin/ld: ./src/network.c:134: undefined reference to `curl_multi_remove_handle'                                                                                                            
/usr/bin/ld: ./src/network.c:137: undefined reference to `curl_easy_cleanup'                                                                                                                   
/usr/bin/ld: ./src/network.c:129: undefined reference to `curl_easy_strerror'                                                                                                                  
/usr/bin/ld: network.o: in function `network_init':                                                                                                                                            
./src/network.c:229: undefined reference to `curl_global_init'                                                                                                                                 
/usr/bin/ld: ./src/network.c:235: undefined reference to `curl_share_init'                                                                                                                     
/usr/bin/ld: ./src/network.c:241: undefined reference to `curl_share_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:242: undefined reference to `curl_share_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:243: undefined reference to `curl_share_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:250: undefined reference to `curl_share_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:251: undefined reference to `curl_share_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:254: undefined reference to `curl_multi_init'                                                                                                                     
/usr/bin/ld: ./src/network.c:259: undefined reference to `curl_multi_setopt'                                                                                                                   
/usr/bin/ld: ./src/network.c:261: undefined reference to `curl_multi_setopt'                                                                                                                   
/usr/bin/ld: network.o: in function `crypto_lock_init':                                                                                                                                        
./src/network.c:56: undefined reference to `CRYPTO_malloc'                                                                                                                                     
/usr/bin/ld: network.o: in function `network_init':                                                                                                                                            
./src/network.c:278: undefined reference to `curl_version_info'                                                                                                                                
/usr/bin/ld: network.o: in function `transfer_blocking':                                                                                                                                       
./src/network.c:318: undefined reference to `curl_easy_setopt'                                                                                                                                 
/usr/bin/ld: ./src/network.c:320: undefined reference to `curl_multi_add_handle'                                                                                                               
/usr/bin/ld: ./src/network.c:323: undefined reference to `curl_multi_strerror'                                                                                                                 
/usr/bin/ld: network.o: in function `transfer_nonblocking':                                                                                                                                    
./src/network.c:335: undefined reference to `curl_multi_add_handle'                                                                                                                            
/usr/bin/ld: ./src/network.c:337: undefined reference to `curl_multi_strerror'                                                                                                                 
/usr/bin/ld: fuse_local.o: in function `fuse_local_init':                                                                                                                                      
./src/fuse_local.c:156: undefined reference to `fuse_main_real'                                                                                                                                
/usr/bin/ld: link.o: in function `Link_to_curl':                                                                                                                                               
./src/link.c:95: undefined reference to `curl_easy_init'                                                                                                                                       
/usr/bin/ld: ./src/link.c:100: undefined reference to `curl_easy_setopt'                                                                                                                       
/usr/bin/ld: ./src/link.c:101: undefined reference to `curl_easy_setopt'                                                                                                                       
/usr/bin/ld: ./src/link.c:103: undefined reference to `curl_easy_setopt'
/usr/bin/ld: ./src/link.c:104: undefined reference to `curl_easy_setopt'
/usr/bin/ld: ./src/link.c:105: undefined reference to `curl_easy_setopt'
[...]

This probably has to do with API changes in libcurl4-openssl-dev ...

[fangfufu]

The problem is not actually due to libcurl4-openssl-dev, right now Debian Testing and Debian Unstable share the same version of libcurl4-openssl-dev. It compiles fine under Debian Testing. Debian Unstable does have a different version of GCC, I don't know if it is because of that.

[fangfufu]

Phew, it took me a while to figure out what was going on. Ok, I fixed it now. The fix is in 97ecbff. I changed the location of $(LDFLAGS). This is because collect2 now has the -as-needed flag. So the linker is now sensitive to the relative position of the object files and the libraries in the command line.

It is basically this error:
https://bugs.launchpad.net/ubuntu/+source/gcc-4.8/+bug/1271468

For more details, have a look at:
https://wiki.ubuntu.com/NattyNarwhal/ToolchainTransition#How_to_Fix_a_Problem
https://wiki.debian.org/ToolChain/DSOLinking#Only_link_with_needed_libraries

[fangfufu]

@jcharaoui , I added some pragmas to suppress -Wunused-function warnings for the thread lock functions OpenSSL. Those functions were needed during Debian Stretch, they became obsolete in Debian Buster. However I am not removing them from the code for now, as people are still using Debian Stretch.

[jcharaoui]

@fangfufu I suggest to remove the functions. I'll create a backport package that includes the required functions (via debian/patches)

[fangfufu]

@jcharaoui I prefer to leave them in, unless there is a technical reason that they should be out. I don't see why we should break backward compatibility. Additionally this software might get used by non-Debian users.

[fangfufu]

The new version had been uploaded to Debian. :-)