Null terminated slices for exec #358

arcnmx · 2016-04-18T20:16:21Z

The goal here is to make allocations optional when passing arguments to the exec family of functions. A Vec<CString> works (and will allocate behind the scenes) while a &[Option<&c_char>] will avoid them as long as you assert that it's null-terminated.

Mostly posting this for feedback on the design; it needs some consideration and probably won't work on old rust versions due to AsRef<CStr>; probably best to wait for #230 and NixString to be figured out for that, and also propagate conversion errors for normal string types.

Notes:

This implementation relies on the option nonzero optimization; is that a stable thing we're allowed to assume exists?
The FnOnce approach was used for the interface instead of normal methods and associated types because it's the only way to use higher ranked lifetimes (well, without forcing the use of a concrete type).

homu · 2016-04-23T19:03:09Z

☔ The latest upstream changes (presumably #357) made this pull request unmergeable. Please resolve the merge conflicts.

arcnmx · 2016-05-03T18:02:04Z

Made some changes to accomodate for the notes:

The nonzero optimization can indeed be relied on as stable.
Moved away from the FnOnce approach. Lifetimes are just added to the methods now, but could still be higher ranked instead (as in A: for<'a> IntoRef<'a, NullTerminatedSlice<&'a c_char>>).

Still waiting on the string story to be figured out before determining the exact conversion traits to be used here instead of AsRef<CStr>.

homu · 2016-09-07T16:27:42Z

☔ The latest upstream changes (presumably #416) made this pull request unmergeable. Please resolve the merge conflicts.

Susurrus · 2017-12-04T04:32:44Z

@arcnmx Would you still be interested in merging this? Given how long this has sit I'd like to either merge this or close this PR.

arcnmx · 2017-12-04T17:17:13Z

I am, as it is nice to be able to allocate the memory for exec ahead of time. Considering that this exposes a new type, I'm still looking for some feedback.

Susurrus · 2017-12-05T03:30:21Z

@arcnmx I'm not too familiar with the exec family of functions and all the background that this starts to touch on. It doesn't help that this commits have no useful descriptions, there are no code comments provided in this PR, and these are no examples or tests added. Many of the high-level comments in this PR description are very good and should make their way into the commits and/or code somehow.

A question that does come up here as well is if there are any other APIs in nix currently that could use the NullTerminatedSlice? I'm wondering if I've seen them here or I'm confusing these with some of the GTK+ APIs.

So it's hard to gauge how "good" of an API this is because I only see the low-level code modifications and there are no tests or examples to gauge how this compares to the old/current way of doing things. That being said, this does seem on the right track for the "ideal" API nix could offer. @arcnmx Do you think you could add some context for these changes through some examples, tests, and comments? I'd also ask that the commits provide some more context as well, but I'm fine for waiting on that a bit.

arcnmx · 2017-12-05T03:53:21Z

Regarding including information in the commit, agreed. The new types and functions certainly could use documentation before actually merging this. Buf if you're unfamiliar with exec, there are a few things to be aware of here:

The posix exec api takes an array of strings, or char** (terminated with (char*)NULL). The current nix api exposes this in such a way that requires allocation of a new Vec to ensure null termination.
A common pattern is fork() followed by exec(), and careful attention must be paid to the operations run after the fork. Particularly, POSIX warns that one should only call "async-signal-safe" functions after a fork, and malloc and friends are not considered such. Real world descriptions and implementations seem to indicate that they can use locks that, after a fork, will deadlock once called again from the forked thread.

Essentially, this PR exposes this "null terminated string array" as a type so that it can be created and allocated beforehand if needed. The side goal here is to still allow impromptu calls to exec() with just a normal slice of C strings.

Susurrus · 2017-12-05T04:00:26Z

@arcnmx Thanks for the additional background, though that was already my takeaway from this. I think the disconnect here is that everything looks fine to me on this, I don't see a downside to this. Removing allocations is always great assuming it doesn't make the API too difficult to use, and I don't have any perspective on what the resultant API looks like here, nor am I aware how the old API was used, without seeing actual example code. Hopefully that makes more sense. And I do think it's worth moving forward with this based on what I'm seeing, and once an example is written up for this I can be 100% on my evaluation of the API this PR proposes.

Susurrus · 2017-12-05T04:01:44Z

Actually, based on your comments, it's actually unsafe the way it's implemented now, because our current API forces a malloc after a fork, yes? So I really don't think you need any feedback here as the way we're doing this now is very wrong. We cannot implement a wrapper around these functions that allocates IIUC.

arcnmx · 2017-12-05T04:08:20Z

Well, hm, again a few main points:

Deadlocks are not necessarily unsafe.
fork is what's actually causing the issue, nix just happens to make it impossible to implement the very common fork + exec pattern properly.

If you're using fork, you must be careful and know exactly what you're doing. nix however adds memory allocation to a function call that is normally just a syscall, which introduces a failure point that should not be there. I'd argue that every single function nix exposes should not transparently allocate memory or do anything other than call the underlying syscall (which was part of the reason we discussed changing how strings and paths are handled), but we must make sacrifices sometimes for ergonomics. impling NixPath for CStr helped with this a lot, but exec stands out as a function that still requires extra allocations to work at all.

Susurrus · 2017-12-05T04:11:17Z

I agree with all your points, I just used the word "unsafe" rather loosely here; what I really meant to say is that nix adds a footgun to the common fork + exec use case, which I think is bad. So let's go ahead and move forward with this PR!

arcnmx · 2017-12-05T04:16:06Z

Alright! So then the question then is, do you see anything wrong with the current design and API/type proposed in this PR? On a high level or even just bikeshedding on type/function names. Documentation is definitely a point that should be addressed.

Susurrus · 2017-12-05T04:34:58Z

Sure, here we go!

Does another crate already implement this FFI, NULL-terminated array functionality we need? I know that GTK+ has their own implemented (this provides a little background), but I don't think that's reusable. I just asked in rust-beginners and someone suggested I post in r/rust. I'd suggest we confirm this doesn't already exist before we implement it. Additionally, should this be implemented in nix, or does this make more sense as a separate crate? Googling for "null-terminated array of pointers rust" yielded what looked to be many people asking about this functionality for FFI use.
NullTerminatedVec isn't used anywhere, but I assume that's because it's how people would actually create the array which would be passed and derefed when calling the function.
Maybe a shorter, more-hip name. Like TerminatedVec or TermVec. I don't think having Null in the name really helps anything and I think the terminator being NULL wouldn't really surprise anyone.

arcnmx · 2017-12-05T05:06:45Z

Hm, good question. I've never seen anything like it, but it certainly does seem like it would be somewhat common in FFI applications. Worth looking around for, though I'd be surprised if you find a primitive that could just be lifted out and used as-is. Should we do up a crate for it, I'm not sure?
*Vec is the owned version, yeah. It goes on the stack/heap/whatever and deref's into the Slice version, which actually impls what the function needs. It's what you use when you need to preallocate before a fork, otherwise if you're fine with allocations you can ignore it and just pass anything Iterator<Item=AsRef<CStr>>.
I like TerminatedVec, it holds an [Option<T>] anyway so it should be pretty obvious what it's terminated by. TermVec is a bit shorthand for my own personal taste, but I'm not that opposed to it.

Susurrus · 2017-12-05T05:11:15Z

Hm, good question. I've never seen anything like it, but it certainly does seem like it would be somewhat common in FFI applications. Worth looking around for, though I'd be surprised if you find a primitive that could just be lifted out and used as-is. Should we do up a crate for it, I'm not sure?

Let's keep it internal then, but make sure it's very modular so it'd be easy to rip out should we find one or should it be useful to others.

Vec is the owned version, yeah. It goes on the stack/heap/whatever and deref's into the Slice version, which actually impls what the function needs. It's what you use when you need to preallocate before a fork, otherwise if you're fine with allocations you can ignore it and just pass anything Iterator<Item=AsRef>.

Is there a reason that we should have an owned version versus making people just create arrays and slices from them? Is there enough need to resize this array that it's worth the whole extra type?

I like TerminatedVec, it holds an [Option] anyway so it should be pretty obvious what it's terminated by. TermVec is a bit shorthand for my own personal taste, but I'm not that opposed to it.

Then lets do TerminatedVec. Also, I'd suggest merging these types into lib.rs as the null_terminated.rs name doesn't make much sense and I think terminated.rs is awkward. lib.rs is generally where common types used through the library live anyways, so let's put these there. Then they're also available at the root level, which is likely what we want.

arcnmx · 2017-12-05T18:03:51Z

Is there a reason that we should have an owned version versus making people just create arrays and slices from them? Is there enough need to resize this array that it's worth the whole extra type?

The type exists for the same reasons why String and CString are used vs just a Vec<u8>: it asserts the invariant that the type is terminated without having to re-check this when it's used. Alternatives for the most part are:

Something that does fn terminate(&Iterator<Item=AsRef<CStr>>) -> (Vec<Option<&char>>, TerminatedSlice). This is impossible because you can't really return two types where one borrows/references the other, so you have to create a new type (TerminatedVec) that can both contain the Vec and impl the necessary trait.
You could make it two-parted, but there are performance and ergonomic concerns against this approach, since you already know it's properly terminated when it's specifically created to be that way:
- let vec = terminate(&Iterator<Item=AsRef<CStr>>) -> Vec<Option<&char>>
- let slice = TerminatedSlice::from_slice(&vec).unwrap()
  - or let slice = unsafe { TerminatedSlice::from_slice_unchecked(&vec) }
We could probably get away with using one type as an enum, or just wrap Cow<[Option<T>]>? I'm actually not sure if that'd be a better or worse approach...

Then lets do TerminatedVec. Also, I'd suggest merging these types into lib.rs

Sounds good.

Susurrus · 2017-12-07T03:19:00Z

I don't know if you needed or wanted feedback to that last comment, but I think we should just keep going with what you have now with TerminatedVec and TerminatedSlice.

arcnmx · 2017-12-08T04:45:39Z

Rebased and addressed a few of the items discussed.

Susurrus

Overall this looks like a pretty nice interface. I would ask that since you're touching these functions that you update the cfgs to match our one-per-line style and improve doc comments for the exec functions you modified.

Also, I'm on mobile, so I can't see the rest of the file, but do we have examples or tests for these functions? It's easiest to tell how good the API is by having compiled examples in our doc comments.

Susurrus · 2017-12-08T15:19:36Z

src/lib.rs

+
+impl<T> TerminatedSlice<T> {
+    /// Instantiate a `TerminatedSlice` from a slice ending in `None`. Returns
+    /// `None` if the provided slice is not properly terminated.


Should this return a Result? This seems like you're using Optipn to return an error

Sure, I can create an empty error type for this.

Susurrus · 2017-12-08T15:22:14Z

src/lib.rs

+}
+
+/// Owned variant of `TerminatedSlice`.
+pub struct TerminatedVec<T> {


This should link to TerminatedSlicr if we're only going to put the documentation on that type.

Hm, should probably duplicate it anyway. Linking to the type is good too though, what's the syntax for that look like?

Susurrus · 2017-12-08T15:22:50Z

src/lib.rs

+impl<T> TerminatedVec<T> {
+    /// Instantiates a `TerminatedVec` from a `None` terminated `Vec`. Returns
+    /// `None` if the provided `Vec` is not properly terminated.
+    pub fn from_vec(vec: Vec<Option<T>>) -> Option<Self> {


Again, this should return a Result type

Susurrus · 2017-12-08T15:24:14Z

src/lib.rs

+    }
+}
+
+impl<T> AsRef<TerminatedSlice<T>> for TerminatedVec<T> {


Did you mean to put this impl up with the Slice implementation code?

Not sure what you mean by this. I tend to follow a type definition with the traits that it impls or provides/enables. You mean this should be moved before the definition of TerminatedVec?

Yes, disregard this comment.

Susurrus · 2017-12-08T15:25:08Z

src/lib.rs

+    }
+}
+
+impl<'a, T: 'a> IntoRef<'a, TerminatedSlice<&'a T>> for &'a TerminatedSlice<&'a T> {


Did you mean to put this and the followng impl up with the Slice implementation code?

Okay I see this one should probably be moved up to follow the TerminatedSlice definition, although the ones above and below it are more related to TerminatedVec.

Susurrus · 2017-12-08T15:26:11Z

src/unistd.rs

-    unsafe {
-        libc::execv(path.as_ptr(), args_p.as_ptr())
-    };
+    try!(path.with_nix_path(|cstr| {


Please use the question mark operator instead of try!

Sure thing! Note that this is inconsistent with the rest of the file, should I just convert them all while I'm at it?

No need to convert any code outside of what you're creating/modifying.

arcnmx · 2017-12-10T17:03:23Z

These functions are currently tested in test/test_unistd.rs, but I'll try to weave in some doc examples for the Terminated* types and functions.

Any comment on impl'ing NixPath for CString? Without it, this change is not backwards compatible (because the prior API required &CString for any paths, contrary to all other nix functions), but maybe that isn't important, or maybe we'll revive the string/path passing discussion and change it all anyway!

arcnmx · 2017-12-10T19:32:53Z

Misc notes:

Formatting still needs fixing:
- ? vs try!()
- not sure what you mean by cfgs but I'm assuming the trait bounds? Is there an example of what the current style is?
Still need to update docs of exec functions. Some of them still mention "slices of CString"
I've included a conversion of the exec tests to use the new API. However, given that the tests/example probably isn't multithreaded, the changes may not strictly be necessary? Feel free to critique and comment on that.
Decision made on whether to include the CString impl of NixPath or not.
The doc example for terminate_cstr has some hidden weirdness to account for passing on android. It should be good though.

Susurrus · 2017-12-11T03:06:40Z

src/lib.rs

+}
+
+/// Coercion of `CStr` iterators into an argument that can be passed to `exec`.
+impl<'a, T: AsRef<CStr> + 'a, I: IntoIterator<Item=T>> IntoRef<'a, TerminatedSlice<&'a c_char>> for I {


Is this something that makes sense to have globally-defined? Or can this be integrated into the function definition instead?

This impl is pretty central to allowing exec to accept either:

&TerminatedSlice (usually actually a TerminatedVec created by terminate())

&[CStr] and similar (specifically any IntoIterator where Item: AsRef<CStr>)

... particularly the latter, which enables the old behaviour of allocate-at-call-site. One slightly less global approach might be to move the IntoRef trait and its impls into the unistd module and possibly give it a name that isn't as generic sounding.

Another option would be to simply remove the old style and require the use of TerminatedVec::terminate_cstr any time exec is called, whether the Vec is being preallocated or not.

I'm a big fan of reducing options and improving consistency. This is why I've been asking for code examples, as I'm not familiar with how people use this API regularly. I want to make sure we know how people use it regularly and our API is easy for the 90% case and possible for the last 10% case. Or something along those lines.

All the examples I see involve a hard-coded array. And in those situations, I think a term_vec!() macro makes a lot of sense; this would look like a regular vec! call and avoid all of the wrapping in Option. But I don't know how people build up these arrays programmatically or if that's ever done in practice. Is that a common operation? I'd think that is handled nicely by the API provided here, but I don't know exactly.

Well, examples are going to use a hard-coded array because they're just examples. Real world usage varies of course, and can be hard-coded in simple cases, or generated from a config file or argv like in the use case that I originally wrote these changes for.

Most uses for exec I can imagine would use data coming from a dynamic source rather than being hard-coded. The convenient no-preallocation variant mostly comes into play when using exec in a program that doesn't fork and just intends to replace the whole process - I'm not sure how common that is, I guess if you're writing a tool like env(1) or something. A single-threaded program that does fork applies too, although that's not an assumption one should make if writing a library that doesn't know what kind of program might be using it.

Susurrus · 2017-12-11T03:06:47Z

src/lib.rs

+    }
+}
+
+impl<T> AsRef<TerminatedSlice<T>> for TerminatedVec<T> {


Yes, disregard this comment.

Susurrus · 2017-12-11T03:07:19Z

src/unistd.rs

-    unsafe {
-        libc::execv(path.as_ptr(), args_p.as_ptr())
-    };
+    try!(path.with_nix_path(|cstr| {


No need to convert any code outside of what you're creating/modifying.

Susurrus · 2017-12-11T03:07:40Z

src/lib.rs

+
+/*
+ *
+ * ===== Null terminated slices for exec =====


Please remove this comment header.

Susurrus · 2017-12-11T03:07:51Z

src/lib.rs

+ *
+ */
+
+use std::ops::{Deref, DerefMut};


Please move this use statements to the top of the file.

Susurrus · 2017-12-11T03:08:12Z

src/lib.rs

+
+/// An error returned from the [`TerminatedSlice::from_slice`] family of
+/// functions when the provided data is not terminated by `None`.
+#[derive(Clone, PartialEq, Eq, Debug)]


Is this not Copy?

Yup, missed that.

Susurrus · 2017-12-11T03:08:28Z

src/lib.rs

+
+/// An error returned from [`TerminatedVec::from_vec`] when the provided data is
+/// not terminated by `None`.
+#[derive(Clone, PartialEq, Eq)]


Missing Copy and Debug.

Debug is defined below this (it's manually implemented because we want to impl it regardless of whether T: Debug or not), and Copy cannot be derived as it contains a Vec.

Whoops, sorry I didn't read that closely enough!

Susurrus · 2017-12-11T03:09:18Z

test/test_unistd.rs

@@ -191,7 +192,7 @@ fn test_initgroups() {
 }

 macro_rules! execve_test_factory(
-    ($test_name:ident, $syscall:ident, $exe: expr $(, $pathname:expr, $flags:expr)*) => (


Since you're modifying this macro, can you create documentation for it so it's known how it's called?

I have no idea how to begin to do this honestly.

You modified the macro, is there no documentation you can offer her to help others at least understand your change to it?

It's a convoluted test generator macro that became more complex with this change (because there are now separate actions to argument initialization vs coercion for use in exec). I can describe some of the intent but it's not going to make all that much sense without reading the whole macro and seeing what goes where.

Anything you know would be a great improvement over what's there now! I'll probably be the one going in there and trying to improve the documentation, so anything you could do to help me out would be appreciated! You at least understand this macro somewhat and I wouldn't think a short blurb and an explanation of your variant would be too difficult to write.

Susurrus · 2017-12-11T03:16:03Z

not sure what you mean by cfgs but I'm assuming the trait bounds? Is there an example of what the current style is?

See https://github.com/nix-rust/nix/blob/master/src/unistd.rs#L888

I've included a conversion of the exec tests to use the new API. However, given that the tests/example probably isn't multithreaded, the changes may not strictly be necessary? Feel free to critique and comment on that.

We have things running in a single thread right now I believe because we limit the test harness to a single process. We'd like to move to tests being run in parallel eventually, so writing thread-safe tests should be attempted at least.

As for the API, we should ideally have examples of both use cases, one where it was preallocated and one where it was not.

Decision made on whether to include the CString impl of NixPath or not.

I think we're going to need to figure out the whole NixPath thing, but it's a little ways down the road right now. I'm not worried about breaking anyone's code since we're pre-1.0, so let's shoot for the nicest API as our goal.

The doc example for terminate_cstr has some hidden weirdness to account for passing on android. It should be good though.

I saw that, I'll give it an extra look through.

Susurrus · 2017-12-11T03:39:30Z

src/lib.rs

+/// An error returned from the [`TerminatedSlice::from_slice`] family of
+/// functions when the provided data is not terminated by `None`.
+#[derive(Clone, PartialEq, Eq, Debug)]
+pub struct NotTerminatedError {


Why isn't this another enum variant within NixError?

That seems like unnecessary bloat to NixError for an error case specific to a single function call that no one will ever actually use or call in 99% of scenarios.

Also chances are if someone does use it, they'll already have created the data to be terminated and actually want the unsafe variant anyway (or if you're avoiding unsafe then failure of the call will be treated as an internal assertion failure that's just abort/unwrap()/unreachable!()). I can't really imagine a scenario where this error would ever be propagated up a method chain or actually used with try!(), which is partially why the original implementation simply used an Option - using an error type is neater, but also kind of overkill?

I'd make the argument that it's less bloat than all the code implementing a new error type, but I understand your point.

As to why this should be an Error type is because the semantics behind that better match what's happening rather than Option. If there is a failure, it should be an error. Options aren't used to denote failure. There is a grey area where it can go either way, but I think this is a clear case for Error.

So let's go ahead and leave this error as is. A goal of mine is to unify the error handling within nix to use the new failure crate, so this will likely get cleaned up along with that change.

Susurrus · 2017-12-11T03:43:37Z

test/test_unistd.rs

@@ -191,7 +192,7 @@ fn test_initgroups() {
 }

 macro_rules! execve_test_factory(
-    ($test_name:ident, $syscall:ident, $exe: expr $(, $pathname:expr, $flags:expr)*) => (


Anything you know would be a great improvement over what's there now! I'll probably be the one going in there and trying to improve the documentation, so anything you could do to help me out would be appreciated! You at least understand this macro somewhat and I wouldn't think a short blurb and an explanation of your variant would be too difficult to write.

arcnmx mentioned this pull request Apr 18, 2016

make execvpe available for all platforms #349

Closed

arcnmx force-pushed the exec-null-terminated branch from 037c46f to ffe8d74 Compare April 25, 2016 14:14

arcnmx force-pushed the exec-null-terminated branch from ffe8d74 to 14c0ad2 Compare May 3, 2016 17:55

Susurrus mentioned this pull request Aug 11, 2017

unistd: add fexecve() #727

Merged

Susurrus added this to the 1.0 milestone Nov 5, 2017

arcnmx added 2 commits December 7, 2017 23:26

impl NixPath for CString (maintains compatibility with exec*)

2170b51

Null terminated slices for exec

9506b74

arcnmx force-pushed the exec-null-terminated branch from 14c0ad2 to 9506b74 Compare December 8, 2017 04:40

Susurrus suggested changes Dec 8, 2017

View reviewed changes

arcnmx added 2 commits December 10, 2017 12:09

execveat should also use NixPath

a613d91

Terminated{Slice,Vec} conversion error type

2d73a55

arcnmx added 3 commits December 10, 2017 14:23

Termination API + example

7976d6d

Adjust exec tests to use TerminatedVec

ea297cc

Fix unused import

fa63f3a

arcnmx added 4 commits December 10, 2017 14:47

execveat should open path in the child

2a7f223

Merge remote-tracking branch 'upstream/master' into exec-null-terminated

c951fb4

fix merge conflict

fed2992

use _exit in fork child example

9667af3

Susurrus suggested changes Dec 11, 2017

View reviewed changes

Null terminated slices for exec #358

Are you sure you want to change the base?

Null terminated slices for exec #358

Conversation

arcnmx commented Apr 18, 2016

homu commented Apr 23, 2016

arcnmx commented May 3, 2016

homu commented Sep 7, 2016

Susurrus commented Dec 4, 2017

arcnmx commented Dec 4, 2017

Susurrus commented Dec 5, 2017 • edited Loading

arcnmx commented Dec 5, 2017 • edited Loading

Susurrus commented Dec 5, 2017

Susurrus commented Dec 5, 2017

arcnmx commented Dec 5, 2017

Susurrus commented Dec 5, 2017

arcnmx commented Dec 5, 2017

Susurrus commented Dec 5, 2017

arcnmx commented Dec 5, 2017

Susurrus commented Dec 5, 2017

arcnmx commented Dec 5, 2017 • edited Loading

Susurrus commented Dec 7, 2017

arcnmx commented Dec 8, 2017

Susurrus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arcnmx commented Dec 10, 2017

arcnmx commented Dec 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arcnmx Dec 13, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Susurrus commented Dec 11, 2017

Choose a reason for hiding this comment

arcnmx Dec 11, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Susurrus commented Dec 5, 2017 •

edited

Loading

arcnmx commented Dec 5, 2017 •

edited

Loading

arcnmx commented Dec 5, 2017 •

edited

Loading

arcnmx Dec 13, 2017 •

edited

Loading

arcnmx Dec 11, 2017 •

edited

Loading