initial version #1

misterbisson · 2016-03-31T21:23:15Z

Current tasks outlined in #1 (comment)

A Dockerized version of sdc-nfs, an NFS v3 server implementation in Node.js.

Start the project and docker exec into the "client" container:

# start the project
docker-compose -f local-compose.yml up -d

# put a test file in the NFS server
docker exec -it nfsserver_nfs_1 touch "/exports/$(date)"

# get a shell in the test client
docker exec -it nfsserver_client_1 bash

Inside the "client" container:

# get the nfs package for Ubuntu
apt-get update && apt-get install -y nfs-common

# make the directory on which we'll mount the NFS volume
mkdir /nfs

# mount the remote NFS volume
mount -t nfs -v -o nolock,vers=3 nfs:/exports /nfs

# list the contents of the NFS volume
ls -al /nfs

The server and client containers need privileged, though I'm hoping we can find a way to avoid that.

tgross · 2016-04-01T12:14:07Z

Dockerfile

+
+# Install our Node.js app
+# $buildDeps will be added for the `npm install`, then removed immediately after
+# the resulting image layer will not include the size of the $buildDeps


nice touch.

misterbisson · 2016-04-01T16:30:29Z

Dockerfile

+ENV CONTAINERBUDDY_CHECKSUM c25d3af30a822f7178b671007dcd013998d9fae1
+ENV CONTAINERBUDDY file:///etc/containerbuddy.json
+
+RUN export CB_SHA1=c25d3af30a822f7178b671007dcd013998d9fae1 \


This SHA is not actually used, but I copypasta'd it from the Jenkins blueprint. @tgross you prefer this one or the other?

Oh it looks like we're have it repeated in the Jenkins blueprint too... it's both CONTAINERBUDDY_CHECKSUM and CB_SHA1. I don't see any reason to have both. I think keeping the checksum in the environment after the installation (via the ENV stanza) doesn't make a whole lot of sense because it's the checksum of the zip file, which we toss out when we're done.

misterbisson · 2016-04-01T18:15:19Z

docker-compose.yml

+      - 111
+      - 2049
+
+client:


docker exec -it nfsserver_client_1 bash

And on nfsserver_client_1:

apt-get update && apt-get install -y nfs-common mkdir /nfs mount -o nolock,vers=3 nfs:/ /nfs

SHA checksum per #1 (comment)

misterbisson · 2016-04-04T02:39:27Z

It seems working. See directions in https://github.com/misterbisson/nfsserver/blob/wip/README.md.

To do:

figure out the /usr/sbin/rpc.idmapd not running noise (see)
~~better health checks for nfs service (see)~~
autopilotpattern-style _env file for the Consul address (see)
more readme
~~add https://github.com/bahamas10/node-manta-sync and optional automatic backups to Manta~~
something else, I'm sure

tgross · 2016-04-04T12:14:49Z

docker-compose.yml

+      - 1892
+      - 2049
+    environment:
+      - CONSUL=consul


This should be - CONSUL unless you're linking; you'll only want - CONSUL=consul for the local-docker-compose.yml.

I was being lazy

I added this as a task in #1 (comment)

misterbisson · 2016-04-04T14:55:11Z

bin/health.sh

+pid=$(pidof /usr/sbin/rpc.idmapd)
+if [ -z "$pid" ]
+then
+    echo "/usr/sbin/rpc.idmapd not running"


This seems to stream errors on Triton without affecting usability.

It does affect the health reporting to Consul, however, so it needs to be removed or fixed.

Strangely enough, this works with no problems on my local test environment, rpc.idmapd is actually running here:

root@5bf58ec1d9c3:/# pidof /usr/sbin/rpc.idmapd 38 root@5bf58ec1d9c3:/# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 18864 10432 ? Ssl 20:48 0:00 /bin/containerbuddy node /opt/nfs/node_modules/sdc-nfs/server.js -f /etc/sdc-nfs.json root 20 0.0 0.0 37092 2860 ? Ss 20:48 0:00 /sbin/rpcbind -w statd 29 0.0 0.3 90352 55940 ? Ss 20:48 0:00 /sbin/rpc.statd root 38 0.0 0.0 23368 220 ? Ss 20:48 0:00 /usr/sbin/rpc.idmapd root 39 0.0 0.1 973044 30608 ? Sl 20:48 0:00 node /opt/nfs/node_modules/sdc-nfs/server.js -f /etc/sdc-nfs.json root 4048 0.1 0.0 20356 3264 ? Ss 21:28 0:00 /bin/bash root 4079 0.0 0.0 17512 2084 ? R+ 21:29 0:00 ps aux root@5bf58ec1d9c3:/#

container on triton:

root@e7dfddacd3c2:/# pidof /usr/sbin/rpc.idmapd root@e7dfddacd3c2:/# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 16507 0.0 1.0 40036 2776 ? Ss 21:22 0:00 /sbin/rpcbind -w root 16522 0.0 11.4 64384 29944 ? Sl 21:22 0:00 node /opt/nfs/node_modules/sdc-nfs/server.js -f /etc/sdc-nfs.json statd 16516 0.0 2.5 43492 6796 ? Ss 21:22 0:00 /sbin/rpc.statd root 1 0.0 4.0 16928 10688 ? Sl 21:22 0:00 /bin/containerbuddy node /opt/nfs/node_modules/sdc-nfs/server.js -f /etc/s root 18108 0.0 1.5 23032 4124 ? Ss 21:25 0:00 /bin/bash root 27305 0.0 1.2 19796 3180 ? R 21:33 0:00 ps aux

Odd that it starts up and runs fine locally but does not run on Triton

Agreed, it's odd. Let's comment out the test for now so it reports healthy, and we'll root-cause it in time.

misterbisson · 2016-04-08T16:35:48Z

@Roxet identified an issue in testing:

I have been testing with bash scripts to download and write 100's of images to the mount with curl and also testing using the "test data" functionality I wrote into the prestart.sh script for the WordPress container.

The first problem comes in when you attempt to write a filename that already exists. I am using 'curl -Lso localfile.ext "http://domain.com/remotefile.ext"' to test this. The default behavior from my testing on the WP container's filesystem appears to be simply overwriting the file. But when I try to overwrite a file on the NFS mount, the NFS container promptly crashes.

Secondly, when I try to import the test data, when the script attempts to import the images the NFS container also crashes. I suspect this crash is related to the fact that the import script attempts to write the files using the original modified timestamp of the file. After the failed attmepts at uploading, i see that it created a 2011 directory in the uploads directory.

both of the crashes give the following error:
nfs_1 | fs.js:882
nfs_1 |   throw new Error('Cannot parse time: ' + time);
nfs_1 |         ^
nfs_1 | Error: Cannot parse time: undefined
nfs_1 |     at toUnixTimestamp (fs.js:882:9)
nfs_1 |     at Object.fs.utimes (fs.js:892:18)
nfs_1 |     at NfsServer.setattr_time (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:245:8)
nfs_1 |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1 |     at f (/opt/nfs/node_modules/sdc-nfs/node_modules/once/once.js:16:15)
nfs_1 |     at NfsServer.setattr_get_atime (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:208:9)
nfs_1 |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1 |     at f (/opt/nfs/node_modules/sdc-nfs/node_modules/once/once.js:16:15)
nfs_1 |     at NfsServer.setattr_get_mtime (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:153:9)
nfs_1 |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1 | Starting rpcbind daemon....
The reference in the error "cannot parse time: +time" plus the fact that the script tries to modify the timestamp on the images lead to my suspicion about the root of the issue.

This error comes from https://github.com/joyent/sdc-nfsserver.

misterbisson · 2016-04-08T17:03:42Z

@Roxet I did the following tests on the us-east-3b DC with no issues:

docker-compose -f local-compose.yml up -d
docker exec -it nfsserver_nfs_1 bash

mkdir /nfs
mount -t nfs -v -o nolock,vers=3 127.0.0.1:/exports /nfs

touch /nfs/thing
touch -t 8412101730 /nfs/thing 
touch -t 3412101730 /nfs/thing 
touch -t 193412101730 /nfs/thing 

curl -Lso /nfs/thing https://github.com/autopilotpattern/nfsserver/pull/1#issuecomment-207505949

I'm going to re-try now from the nfs client.

misterbisson · 2016-04-08T17:13:22Z

docker exec -it nfsserver_client_1 bash

apt-get update && apt-get install -y nfs-common
mkdir /nfs
mount -t nfs -v -o nolock,vers=3 nfs:/exports /nfs

touch /nfs/thing2
touch -t 8412101730 /nfs/thing2
touch -t 3412101730 /nfs/thing2 
touch -t 193412101730 /nfs/thing2 
touch: setting times of '/nfs/thing2': Value too large for defined data type

curl -Lso /nfs/thing2 https://github.com/autopilotpattern/nfsserver/pull/1#issuecomment-207505949

I got this error for touch -t 193412101730 /nfs/thing2:

touch: setting times of '/nfs/thing2': Value too large for defined data type

...but, that didn't result in any errors on the server.

@Roxet can you help me find a repeatable way to demonstrate this bug? Do you know what time/date it was attempting to set?

jasondewitt · 2016-04-08T19:01:02Z

@misterbisson strange, testing this on my copy of this repo, with the nfs server container and client container, I used the same commands and I get no error until I try to overwrite the file:

root@13ffc84b304b:/# apt-get update && apt-get install -y nfs-common curl   
root@13ffc84b304b:/# mkdir nfs
root@13ffc84b304b:/# mount -t nfs -v -o nolock,vers=3 nfs:/exports /nfs
mount.nfs: timeout set for Fri Apr  8 18:55:47 2016
mount.nfs: trying text-based options 'nolock,vers=3,addr=172.17.0.3'
mount.nfs: prog 100003, trying vers=3, prot=6
mount.nfs: trying 172.17.0.3 prog 100003 vers 3 prot TCP port 2049
mount.nfs: prog 100005, trying vers=3, prot=17
mount.nfs: portmap query retrying: RPC: Program not registered
mount.nfs: prog 100005, trying vers=3, prot=6
mount.nfs: trying 172.17.0.3 prog 100005 vers 3 prot TCP port 1892
root@13ffc84b304b:/# touch /nfs/thing2
root@13ffc84b304b:/# touch -t 8412101730 /nfs/thing2
root@13ffc84b304b:/# touch -t 3412101730 /nfs/thing2 
root@13ffc84b304b:/# touch -t 193412101730 /nfs/thing2 
root@13ffc84b304b:/# curl -Lso /nfs/thing2 https://github.com/autopilotpattern/nfsserver/pull/1#issuecomment-207505949

As soon as I enter that last curl command I see the errors come in the nfs containers' logs:

nfs_1    | {"name":"server.js","hostname":"2b363718923e","pid":42,"component":"RpcServer","procedure":"mnt","xid":3588741937,"level":30,"msg":"mount(/exports) from (172.17.0.4): done -> 0510e639-bf5d-4f91-bd99-94ce33d8b455","time":"2016-04-08T18:53:47.278Z","src":{"file":"/opt/nfs/node_modules/sdc-nfs/lib/mount.js","line":119},"v":0}
nfs_1    | 
nfs_1    | fs.js:882
nfs_1    |   throw new Error('Cannot parse time: ' + time);
nfs_1    |         ^
nfs_1    | Error: Cannot parse time: undefined
nfs_1    |     at toUnixTimestamp (fs.js:882:9)
nfs_1    |     at Object.fs.utimes (fs.js:892:18)
nfs_1    |     at NfsServer.setattr_time (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:245:8)
nfs_1    |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1    |     at f (/opt/nfs/node_modules/sdc-nfs/node_modules/once/once.js:16:15)
nfs_1    |     at NfsServer.setattr_get_atime (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:208:9)
nfs_1    |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1    |     at f (/opt/nfs/node_modules/sdc-nfs/node_modules/once/once.js:16:15)
nfs_1    |     at NfsServer.setattr_get_mtime (/opt/nfs/node_modules/sdc-nfs/lib/nfs/setattr.js:153:9)
nfs_1    |     at next (/opt/nfs/node_modules/sdc-nfs/node_modules/oncrpc/lib/server.js:318:22)
nfs_1    | 2016/04/08 18:54:39 exit status 8
nfsserver_nfs_1 exited with code 0

Could this be a problem with my Docker version? Also, are you guys testing on mac? I'm running Ubuntu, could be a host issue...

 jason  ~  docker -v
Docker version 1.9.0, build 76d6bc9
 jason  ~  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.2 LTS
Release:    14.04
Codename:   trusty

I'm booting these guys up on triton now to remove that possibility form the test.

misterbisson · 2016-04-08T19:42:06Z

Also, are you guys testing on mac? I'm running Ubuntu, could be a host issue...

All my testing above was on Triton. I'll re-try locally.

jasondewitt · 2016-04-08T19:48:57Z

i'm running on triton now too, not getting the same problem. I did see this error come up once when running the curl test, but I only got it once and can't reproduce

nfs_1 | {"name":"server.js","hostname":"90d6864b76a1","pid":75681,"component":"RpcServer","procedure":"write","xid":1976409319,"level":40,"err":{"message":"EBADF, write","name":"Error","stack":"Error: EBADF, write","code":"EBADF"},"msg":"write: failed","time":"2016-04-08T19:42:57.797Z","src":{"file":"/opt/nfs/node_modules/sdc-nfs/lib/nfs/write.js","line":26},"v":0}

I'm going to get my WordPress compose stuff in shape to run on Triton and test it there.

misterbisson · 2016-04-17T19:13:08Z

Moving some todos to future issues:

Health checks: Improved health checks #2
Manta backups: Add sync to Manta and (optional) automatic backups #3

tgross · 2016-04-18T13:00:59Z

I ran into something a bit odd here with the ownership of /exports. I think when we're creating the volume in the Dockerfile that something is happening with user ID mappings:

docker exec -it nfs_nfsserver_1 /bin/bash
root@9ccad5c12507:/# ls -lah /exports/
total 2.0K
drwxr-xr-x  2 root uucp  2 Apr 18 12:43 .
drwxr-xr-x 24 root uucp 24 Apr 18 12:43 ..

This was on Triton so I was using the autopilotpattern/nfsserver:latest image. If I build locally I don't see this UUID. It's just root:root.

tgross · 2016-04-18T13:05:28Z

docker-compose.yml

+
+nfsserver:
+    image: autopilotpattern/nfsserver
+    privileged: true


This is pretty bad juju on Docker-on-Linux. Can we make this more fine-grained with --cap-add or --cap-drop or --device? (ref https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)

See #1 (comment)

This is not recommended for production use

Server and client containers need privileged on Linux hosts (though not on Triton, which supports this securely). This may not be a solvable problem. Docker volume drivers are probably the best recommended work around. On Triton, RFD26 will provide network shared filesystems to Docker containers using Docker volume syntax.

For now, consider this an experiment.

Aye, works for me.

misterbisson · 2016-04-18T14:17:01Z

README.md

+This repo Dockerizes [sdc-nfs](https://github.com/joyent/sdc-nfs), an NFS v3 server implementation in Node.js. This is intended to allow use NFS in projects without requiring kernel NFS support or privileged access, but that is unfortunately not true. 
+
+## This is not recommended for production use


@tgross I added this note explaining privileged

misterbisson added 4 commits March 31, 2016 14:24

first try

a3021b6

Doesn't like Alpine or Node.js 4...

fa36b4a

node 0.10 works

b9e7c9b

smaller, sweeter

ca79df8

misterbisson force-pushed the wip branch from ca28de6 to ca79df8 Compare March 31, 2016 21:38

tgross reviewed Apr 1, 2016
View reviewed changes

misterbisson added 3 commits April 1, 2016 08:59

add compose yaml

d6c15f8

nfs-common is required

4bf1307

expose the nfs ports, duh

5644d6b

misterbisson reviewed Apr 1, 2016
View reviewed changes

misterbisson added 2 commits April 1, 2016 11:12

remove the portmap section entirely to use internal portmapper

2728da2

rearranging deck chairs

fdb57ba

misterbisson reviewed Apr 1, 2016
View reviewed changes

misterbisson added 11 commits April 1, 2016 14:24

vim for debugging, damnit

aae9737

trying the host portmapper again

e78293e

add containerbuddy

f927e65

local compose

7d585cf

polish

27651c1

pushed the image to Docker Hub

ae502b7

readme

8e3ebe7

there shall be only one

2cb5aaa

SHA checksum per #1 (comment)

polish

b6a81e1

working

64bebfd

spit and polish

664f772

tgross reviewed Apr 4, 2016
View reviewed changes

misterbisson changed the title ~~work in progress~~ initial version Apr 4, 2016

misterbisson reviewed Apr 4, 2016
View reviewed changes

misterbisson mentioned this pull request Apr 5, 2016

in progress autopilotpattern/wordpress#3

Merged

update for ContainerPilot

d756abf

tgross reviewed Apr 18, 2016
View reviewed changes

add experiment note

879d9f3

misterbisson reviewed Apr 18, 2016
View reviewed changes

misterbisson merged commit 275fbb4 into master Apr 22, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

initial version #1

initial version #1

misterbisson commented Mar 31, 2016 •

edited

Loading

tgross Apr 1, 2016

misterbisson Apr 1, 2016

tgross Apr 1, 2016

misterbisson Apr 1, 2016

misterbisson commented Apr 4, 2016 •

edited

Loading

tgross Apr 4, 2016

misterbisson Apr 4, 2016

misterbisson Apr 4, 2016

misterbisson Apr 4, 2016

jasondewitt Apr 13, 2016

misterbisson Apr 14, 2016

misterbisson commented Apr 8, 2016

misterbisson commented Apr 8, 2016

misterbisson commented Apr 8, 2016

jasondewitt commented Apr 8, 2016

misterbisson commented Apr 8, 2016

jasondewitt commented Apr 8, 2016

misterbisson commented Apr 17, 2016

tgross commented Apr 18, 2016

tgross Apr 18, 2016 •

edited

Loading

misterbisson Apr 18, 2016

tgross Apr 18, 2016

misterbisson Apr 18, 2016

+                    - 111
+                    - 2049
+              client:

		This repo Dockerizes [sdc-nfs](https://github.com/joyent/sdc-nfs), an NFS v3 server implementation in Node.js. This is intended to allow use NFS in projects without requiring kernel NFS support or privileged access, but that is unfortunately not true.

		## This is not recommended for production use

initial version #1

initial version #1

Conversation

misterbisson commented Mar 31, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

misterbisson commented Apr 4, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

misterbisson commented Apr 8, 2016

misterbisson commented Apr 8, 2016

misterbisson commented Apr 8, 2016

jasondewitt commented Apr 8, 2016

misterbisson commented Apr 8, 2016

jasondewitt commented Apr 8, 2016

misterbisson commented Apr 17, 2016

tgross commented Apr 18, 2016

tgross Apr 18, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This is not recommended for production use

Choose a reason for hiding this comment

Choose a reason for hiding this comment

misterbisson commented Mar 31, 2016 •

edited

Loading

misterbisson commented Apr 4, 2016 •

edited

Loading

tgross Apr 18, 2016 •

edited

Loading