rmfuse on OSX #2
Comments
|
Urgh. I was afraid of something like this. I was hoping that a version number > 3.0 on MacFuse indicated that it supported FUSE3, but apparently not. From those issues, I don't hold much hope of MacFuse ever making it to FUSE3. I don't think there's anything I'm doing that requires FUSE3 specifically; that requirement is coming from the fact that I chose to use pyfuse3, which seemed to be the most active and best documented library. There were a bunch of other bindings I found out there, but the only other one I seriously considered was python-fuse. This is for FUSE2, but I don't know whether it works with MacFuse. If someone would like to check and let me know, we could see if converting to the python-fuse library makes sense. (A simple test would be to All that said, this particular error was because the pyfuse3 module needs to be compiled during installation, and it couldn't find the fuse3 headers. The fuse-python module is the same way, so I don't know if this would also hit the same problem. |
|
Yes I think libfuse2 should work with osx. I tested the example script that you provided (after I successfully installed fuse-python). I get an error, which seems to be related to the example script, rather than fuse itself. However, since import fuse
from fuse import Fuseworks. I have no doubt that using libfuse2 would fix the problem for OSX. However, I suppose that libfuse3 is more stable.... Its a pity that macfuse did not update libfuse.... |
|
Thanks for checking. It appears that pyfuse3 forked off from python-llfuse some time ago. That project is now marked as undeveloped, but the API is still pretty similar. It has some hints on building solutions compatible with both libraries. I'll take a look and see if that's feasible here. |
|
Good news -- it seems to work. See #6 for a branch that implements support for llfuse. If you could try it out and let me know if it works with MacFuse, it'd be much appreciated. You'll need llfuse ( |
|
That sounds awesome! I will test it today! |
|
It installed successfully! One needs to allow a kernel extension and reboot. I messed up the onetime code and now get the error: Seems to be unrelated to rmfuse. |
|
That is a bug in RMfuse--it stores file metadata in a cache database to avoid hitting the API too often. But I didn't make sure the |
|
thanks for getting back! I started (rmfuse) schneider@Aarons-MacBook-Pro ~ % rmfuse ~/remarkable -v
DEBUG:llfuse:Initializing llfuse
DEBUG:llfuse:Calling fuse_mount
DEBUG:llfuse:Calling fuse_lowlevel_new
DEBUG:llfuse:Calling fuse_session_add_chan
DEBUG:llfuse:Calling fuse_session_loopIt doesn't finish the session. A new window yields: (base) schneider@Aarons-MBP ~ % cd remarkable
(base) schneider@Aarons-MBP . % ls
ls: .: Function not implemented
(base) schneider@Aarons-MBP . % open .
2021-02-14 12:52:14.104 open[1833:36366] CFURLCopyResourcePropertyForKey failed because it was passed an URL which has no scheme
No application knows how to open . (Error Domain=NSOSStatusErrorDomain Code=-10814 "kLSApplicationNotFoundErr: E.g. no application claims the file" UserInfo={_LSLine=1483, _LSFunction=runEvaluator}).
(base) schneider@Aarons-MBP . % cp ~/Desktop/Chapter_3.pdf ~/remarkable
cp: /Users/schneider/remarkable: Function not implementedDon't get confused by |
|
Second question first: the tokens are stored in As for the first problem: it appears that your system is trying to call a filesystem method that isn't implemented in RMfuse yet. (There are quite a few, and I implemented them as I found they were needed in testing.) It's not clear to me which one it is, but if you run Thanks! |
|
Thanks! Here we go: |
|
I see a lot of failed calls to statfs, which hadn't been implemented. I've added a simple implementation in #6. Can you give it a try. Updated sdist file attached, if that helps. |
|
Awesome! it works for copying stuff from the fuse drive to the local machine. But right now its giving me an error, when I try to copy a pdf to the fuse drive. Here is a new log: |
|
The error seems to be that RMfuse cannot determine that the file you're trying to upload is a PDF or EPUB file. Which type is it? (Note that if it's not one of these, the upload should fail.) If you're able to send me the file, I can take a look. If not, just the beginning of the file would be enough for me to see if RMfuse should be able to recognize it. You can run |
|
Hi, I tested it with the same file that I downloaded from the remarkable. It was definitely a pdf file. Here is again the python error trace when I try to Here is another error which I get upon renaming a file: Here is the debug.pdf that I copied from the rmcloud to my desktop. And which I tried to upload back to the rm. On the rm this file is called Thanks! |
|
Thanks. I think I missed the tracebacks into the logs above. The good new is, both of these are simple bugs. I've pushed fixes; give it a try. (I swear we're getting closer, even if it doesn't seem that way!) |
|
Thanks again for debugging! Renaming and deleting files seems to work now. Moving files (either by shell command or by finder) does not yet work. Both give always errors. Edit: Is there a reason why rmfuse runs in the foreground by default? Its not a problem to do something like |
|
I'm not convinced that those errors are a problem. The LOOKUP is probably result of something on your system looking for a file that doesn't exist, perhaps some metadata directory. The RELEASEDIR isn't implemented, since RMfuse doesn't have anything to do to release a directory. But that shouldn't cause problems. (We can try adding a stub if we can't figure out anything else, but I suspect it wouldn't really help.) What's the failure mode of moving the file? Does it crash? If so, please provide the traceback. Does it hang? How long did you wait? Does nothing happen? Does this happen when moving files within the RMfuse mount point, moving a file from your system to RMfuse, or moving a file from RMfuse to your system? |
|
sorry for the poor Bugreport. Let me try to improve on your questions: It oddly depends on the file (and its size). I tested four files:
Copying using the terminal succeeds for the smaller documents ( In terminal that is in control of However the two larger documents do not succeed: in rmfuse window:
rmfuse does not crash nor hang.
a couple of minutes. I also checked with rmapi if the file was copied but just not listed.
Moving a file inside rmfuse and moving a file from rmfuse to system gives no error. Only moving files from local to rmfuse. |
rschroll
added a commit
that referenced
this issue
Feb 21, 2021
It seems possible that this is a source of trouble in #2. So we'll keep a record of all documents being uploaded right now, and check that if we don't find a document in the cloud or in the buffers. This could go sideways very quickly if we try to do something with this document. But hopefully in practice it will only be setting attributes or something similarly innocuous.
|
Thanks, that's very informative. I just don't quite know what it all means. :) The errors about the extended attributes are expected -- RMfuse doesn't implement them right now. (It might be possible to stuff them into the document metadata at some point.) But it is weird that the error on setting them is different in the two cases. If they get annoying, we could add an option to RMfuse that would pretend to set the attributes, which should get rid of that message. I'm not sure what's going wrong with the "Not a PDF or EPUB file" error. My best guess is that there's some metadata file that the Mac is trying to create / copy along with the PDF file. The fact that it's happening in both successes and failures suggests that it's not a fatal problem. The worrisome bit is the "Attempt to get non-existent Item". That means something is going wrong in the upload process. I wonder if we have a race condition -- something is trying to do something to a file while it's in the process of being uploaded. This would slip in a gap of awareness of RMfuse -- it can't find the document in the set of editable new-created documents, since we closed it out to upload. But it also doesn't see it in the list of documents from the Cloud, because the upload hasn't finished. (This would explain why it's only happening to larger files.) I've built a new version with some more debugging output and an attempt to handle the above case. Please give it a try (with |
Open
|
Thanks for the explainations! Copying Molliere_thesis or ms2 to rmfuse using finder (still same error code -43 in finder): The same from command line (only the last couple lines...): In contrast here we have ms2.pdf which works: From what I can see: The difference between a finder copy and a terminal copy seems to be everything in between the first line and the last few lines. |
|
Thanks. I feel like we're inching towards understanding. The thing that has
me worried is this:
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis)
This means that RMfuse couldn't figure out that the file is a PDF document.
Therefore, it couldn't upload the file, and that's why nothing's working.
It looks like there may be two different failures, unfortunately. With
finder, it appears that no data was written to the file. I'm not sure why
that is. (Maybe finder planned on re-opening the file to write the contents
later?) But it makes sense that it failed. But from the command line, we
can see a lot of data did get written, so I'm not sure what went wrong
there.
RMfuse assumes that a PDF file starts with '%PDF'. Can you check whether
this is the case for your problematic files? (The 'head' command is useful
for reading the first n bytes of a file.) If it's not, can you include the
first couple of KB of those files so I can take a look at what's going on
there?
…On Sun, Feb 21, 2021 at 11:23 PM Aaron David Schneider < ***@***.***> wrote:
Thanks for the explainations!
Copying Molliere_thesis or ms2 to rmfuse using finder (still same error
code -43 in finder):
DEBUG:rmfuse.fuse:Created Molliere_thesis for b'Molliere_thesis.pdf', with inode 16 and ID 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289
DEBUG:rmfuse.fuse:Releasing inode 16
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis)
ERROR:root:Attempt to get non-existent Item 9dc8cf3f-17ec-40e0-80e1-f4b61ea41289
The same from command line (only the last couple lines...):
(...)
DEBUG:rmfuse.fuse:Writing to 18 at 30257152, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30261248, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30265344, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30269440, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30273536, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30277632, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30281728, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30285824, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30289920, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30294016, length 4096
DEBUG:rmfuse.fuse:Writing to 18 at 30298112, length 755
DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:setattr called on <Document "Molliere_thesis">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Created ._Molliere_thesis for b'._Molliere_thesis.pdf', with inode 19 and ID 870c59c6-47aa-42a1-8599-291f5e2815e9
DEBUG:rmfuse.fuse:Writing to 19 at 0, length 4096
DEBUG:rmfuse.fuse:Releasing inode 19
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._Molliere_thesis)
DEBUG:rmfuse.fuse:Releasing inode 18
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was Molliere_thesis)
ERROR:root:Attempt to get non-existent Item 4a4313ac-4c86-493d-9f5b-2378666aa553
In contrast here we have ms2.pdf which works:
DEBUG:rmfuse.fuse:Created ms2 for b'ms2.pdf', with inode 20 and ID 6e376b45-4c4b-4e4b-a5b1-34e9679da343
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33152
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Writing to 20 at 0, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 4096, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 8192, length 4096
(...)
DEBUG:rmfuse.fuse:Writing to 20 at 950272, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 954368, length 4096
DEBUG:rmfuse.fuse:Writing to 20 at 958464, length 2067
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:setattr called on <Document "ms2">
DEBUG:rmfuse.fuse: Attempting to set mode to 33188
DEBUG:rmfuse.fuse: No changes made
DEBUG:rmfuse.fuse:Created ._ms2 for b'._ms2.pdf', with inode 21 and ID 539b5949-8baa-4336-9c60-5febbb25dfe3
DEBUG:rmfuse.fuse:Writing to 21 at 0, length 4096
DEBUG:rmfuse.fuse:Releasing inode 21
ERROR:rmfuse.fuse:Error: Not a PDF or EPUB file (file was ._ms2)
DEBUG:rmfuse.fuse:Releasing inode 20
DEBUG:rmcl.items:Getting details for <Document "ms2">
DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531
ERROR:root:Attempt to get non-existent Item 539b5949-8baa-4336-9c60-5febbb25dfe3
*From what I can see:*
The difference in cp from terminal between Molliere and ms2 boils down to
these two lines missing for Molliere_thesis:
DEBUG:rmcl.items:Getting details for <Document "ms2">
DEBUG:rmcl.items:Details for <Document "ms2">: type pdf, size 960531
The difference between a finder copy and a terminal copy seems to be
everything in between the first line and the last few lines.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACZ2HZQYXJG545Q2WXCKMLTAIA7DANCNFSM4XISMUSA>
.
|
|
Thanks for your quick reply! Here is the head output of Molliere thesis: I uploaded a test file (an old presentation I gave during my undergrad): We could use this file as a benchmark for this issue since this one fails too and I can share it. |
|
The bad (or maybe good?) news is that that NASA PDF file works just fine on my test system. This suggests to me that the Mac is doing something I didn't anticipate when writing the file. Can you check through the 'Writing to X at Y, length Z' messages? Does Y start at zero and always increase? Or does it jump around. (On my system, it increases steadily, but if it doesn't I can imagine that it might break things. Though I don't see how it can break the beginning of the file.) The only other thought I have is to look at the actual values RMfuse finds in the file. If you're willing to get your hands dirty, try adding a line to rmfuse/fuse.py. Right after the line Try adding This will print out the first KB of the file as RMfuse got it, and we can see if it's different from what it should be. If you're not comfortable mucking about like this, I can create an instrumented version with this and other debug output later. |
|
I just tested your second idea with the Nasa file. Upon I get the following output in the rmfuse window: It looks like it is trying to copy two files. The second one has only I might be mistaken, but is that really the content of the pdf??? But I need to admit that I do not know much about binary files... other than that they are nasty to deal with ... |
|
Yeah, it looks like the file it is trying to upload is full of null bytes. That ain't right. (As we saw above, the PDF file should start with '%PDF'.) Unfortunately, I have no idea why that would be the case. Perhaps the OS decided it would fill the file with zeros first, and then write the actual content? But I don't know why that would be the case, or why that would only happen with large files. Another possibility would be a bug with llfuse or Macfuse. But again, I don't know why that would only affect large files. I'll have to puzzle over this for a while. In happier news, those |
|
No hurry 👌
I can confirm. |
|
No brilliant thoughts so far. Just to confirm: Other than uploading large files, things seem to be working pretty well? I'm thinking that maybe I should merge #6 into master and cut a new release, with the hope that additional users will help us figure out what's going wrong. How does that sit with you? |
|
I've added MacOS install directions, based on your descriptions here. Can you check that I got the correct? https://github.com/rschroll/rmfuse/blob/release-0.2.0/README.md#macos Note that if you want to try this out yourself, you'll need to use the Thanks! |
|
yes that sounds good! |
|
The new version doesn't start on osx. The reason seems to be the typo: Also: I suppose |
|
On Sun, Feb 28, 2021 at 10:55 PM Aaron David Schneider < ***@***.***> wrote:
The new version doesn't start on osx.
The reason seems to be the typo: noappledouble instead of nodoubleapple.
D'oh!
I can't install it the way you suggest it:
(rmfuse) ***@***.*** ~ % pip install rmfuse[llfuse]
zsh: no matches found: rmfuse[llfuse]
You'll need to add the `--pre` option, to look for pre-release versions in
pip.
|
rschroll
added a commit
that referenced
this issue
Mar 2, 2021
|
I'll close this issue for now, and we can keep working on the large file upload problem in #18. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I really love the idea of
rmfuseand I would love to use it on my Mac.It seems that that
pyfuse3is not supported on OSX:macfuse/macfuse#390
libfuse/pyfuse3#29
This is the error I get when doing
pip install rmfuseThe text was updated successfully, but these errors were encountered: