Spec for the NEW sandboxed Local Storage - Feedback appreciated

[gliechtenstein]

One of the most frequently requested features is local key/value storage. This is a sensitive problem since it's directly related to security. Browsers have sandboxing for many good reasons. This is why I've been delaying making a decision on this until I've come up with a couple of ideas.

Now I have, and wanted to share. Note that none of the ideas involve having a single global storage, but involves having a local storage sandbox per each URL and accessing them from one view to another.

Here are the two:

1. Readonly exports

If you are aware of javascript module system you can think of this as a similar method. We would introduce two new actions $state.export and $state.import. Users would need to first visit a URL to "export" all the values it wants to export, and then once that's done, other views can reference the value using $state.import. If you try to $state.import before it's been exported, it will not return the value;

I actually took a shot at a quick prototype over at $state branch The code is commented so it should be self-explanatory how it should work.

• Pros: it's pretty simple (conceptually as well as implementation-wise) and intuitive. Easy to implement.
• Cons: the states are "readonly" from other views. You can write to the state of any URL ONLY from itself.

An example code:

Exporting:

    {
        "type": "$state.export",
        "options": {
            "username": "Ethan",
            "email": "ethan.gliechtenstein@gmail.com"
        },
        "success": {
            "type": "$close"
        }
    }

Importing

     {
        "type": "$state.import",
        "options": {
            "db": "https://www.jasonbase.com/things/3nf.json"
        },
        "success": {
            "type": "$set",
            "options": {
                "db": "{{$jason.db}}"
            }
        }
     }

It works pretty similarly to how Javascript module system works. you can export and other modules can import to use them, but they are readonly (the similarity to Javascript module system is another reason why this feels simpler)

But like I said, not being able to write has some drawbacks, which is why I have another idea:

2. Read/Write state variables

Here, you not only can read from another URL's state, but also can write to it. Here the API would consist of: $state.set and $state.get.

Since being able to write to another view's sandbox is a big deal in terms of security, we also need another security measure. In this case we will also need to explicitly state the public state variables so that only those state variables can be set from outside of the view. This may look something like this:

{
  "$jason": {
    "head": {
      "state": ["username", "email"],
      ...
    }
  }
}

Basically we are stating that anyone can set the state variable for this URL but ONLY the "username" and "email" variables.

I've attached below a full example of what it would look like. But before we go on, just a quick explanation on how this works:

1. abc.json is the master view which transitions to 123.json
2. User can select an option from 123.json
3. When the user selects an item, it sets the abc.json's selected state variable via $state.set, and returns via $back action.
4. When we return to abc.json, the $show event triggers $state.get and renders the view based on the state we've just set.

abc.json

{
  "$jason": {
    "head": {
      "state": ["selected"],
      "actions": {
        "$show": {
          "type": "$state.get",
          "options": {
            "db": "https://www.jasonbase.com/things/abc.json"
          },
          "success": [{
            "{{#if ('db' in $jason) && ('selected' in $jason.db)}}": {
              "type": "$set",
              "options": {
                "selected": "{{$jason.db.selected}}"
              },
              "success": {
                "type": "$render"
              }
            }
          }, {
            "{{#else}}": {
              "type": "$set",
              "options": {
                "selected": "Nothing selected"
              },
              "success": {
                "type": "$render"
              }
            }
          }]
        }
      },
      "templates": {
        "body": {
          "sections": [{
            "header": {
              "type": "label",
              "text": "{{$get.selected}}"
            },
            "items": [{
              "type": "label",
              "text": "Select Items",
              "href": {
                "url": "https://www.jasonbase.com/things/123.json"
              }
            }]
          }]
        }
      }
    }
  }
}

123.json

{
  "$jason": {
    "head": {
      "data": {
        "items": [{
          "text": "Item 1"
        }, {
          "text": "Item 2"
        }, {
          "text": "Item 3"
        }]
      },
      "templates": {
        "body": {
          "sections": [{
            "items": {
              "{{#each items}}": {
                "type": "label",
                "text": "{{text}}",
                "action": {
                  "type": "$state.set",
                  "options": {
                    "selected@https://www.jasonbase.com/things/abc.json": "{{text}}"
                  },
                  "success": {
                    "type": "$back"
                  }
                }
              }
            }
          }]
        }
      }
    }
  }
}

• pros: more flexible
• cons:
  1. more complex, which means there can be some unforeseen edge cases.
  2. This one's more important: Because we rely on the "state" declaration on the view in order to determine what state variable is read/writeable, every time we access $state.get, we need to query the the state's owner URL (one network/file request) to check its $jason.head.state attribute to make sure the attribute has been declared accessible. (The first solution I suggested above doesn't have this problem since the variable will simply be empty if it hasn't been already exported)

---

Feedback appreciated

I wanted to share here because I'm still trying to decide on this since people have been requesting this either directly or indirectly (a lot of the problems people talk about on the forum can be made easier by supporting this, for example almost all bugs/issues related to tab bar or $params will become irrelevant when we support this since we can deprecate $params and just use the $state instead, which is much more robust and flexible).

I would appreciate any kind of feedback on either of the two. Especially if you have a better idea or improvement, or any security holes I missed, please feel free to share.

Thanks!

[j2l]

Very interesting!
Do you have an example where 2 solves a problem 1 can't?
If not, I prefer 1.

[steve21124]

@j2l Yes. flexibility come with complexity. Prefer simple 1 unless no 2 has real benefit.

[drwasho]

Finally got to read this. This looks great and it would solve some of the problems I've been facing!

[gliechtenstein]

@drwasho the reason I came up with option 2 was because you mentioned to me how the first approach wouldn't work for you. Maybe you can weigh in on the conversation on whether option 2 actually solves your problem, and if so, tell us why that works better than option 1 for you, so we can make a better decision moving forward?

[drwasho]

For what I'm building, there were 2 reasons why I decided not to use the option 1 approach:

1. I had no easy way to clear the state in other views other than to have the user go into those views and either: a) have them remove whatever data persisted there, or b) automatically clear the state every time you open that view (which was a bad idea)

2. From a data handling POV, it was easier to access and clear variables that are locally scoped, at the price of more complexity in the view

And 2 is the big one. This is particularly relevant when you have views that are submitting a complex form, like an ecommerce listing.

For example, imagine I have a parent view where I set most of the listing data, but I also have a child review where I set the shipping options. If I set shipping options in the child view, then when I go to create a new listing at another time, the shipping options from the previous listings will persist in state. The same would be true for listing options or any other detailed complex data that needs its own child view. Ultimately this means means I’ll end up creating listings with the details of old listings unless some very bad UI/UX is introduced.

Alternatively I prefer to have a complex view + mix-ins (which make it manageable) where all the variables are local and easy to manipulate/clear.

So I think option 2 is the best here, and I'd add that it would be even better if we had an $state.reset action too.

However, my real preference is to have a global key:value storage that is read/write accessible from any view (actually configurable, as in X view can read and Y view can read/write).

This does become a bit of a security headache when you have views that are remotely hosted, but for those of us creating production apps with locally stored views, it is less of a problem.

[j2l]

@drwasho, it makes sense.
thanks!
May I ask how you manage the child view of a product? With templates only or views? If views, how do you manage to have them offline?

[Joshfindit]

Two comments:

• Second vote here for the ability to have a global storage per app, with all views inside the app being able to access it, but I do also like this idea of protecting some values behind the export/import scheme.
• In the data-per-url scheme, would it be beneficial to think of only allowing certain other urls/views to access protected keys?

[gliechtenstein]

In the data-per-url scheme, would it be beneficial to think of only allowing certain other urls/views to access protected keys?

@Joshfindit Yes that was the idea. In case of the second spec (which I am leaning towards), there's an attribute called state, which will just be an array of accessible attributes for the initial release, but eventually it will contain something like a key value pair of security info such as:

state: [{
  "username": {
    "read": "*",
    "write": "https://jasonbase.com/things/3nf.json"
  }
}]

In this case the username attribute is readable from all urls but only writable by the specified URL, or something like that. But I don't want to think too deep into this yet since the first version isn't even out yet. Probably will go with the "allow all" approach as the first version

[gliechtenstein]

OK I made some updates. Here's the new syntax. This will probably be the final version that will roll out unless someone finds a bug or weird cases I haven't thought of.

Before I explain, you can try out a working demo at https://jasonbase.com/things/amWz

Basically the example creates a "chatroom" but everything is stored locally. Note that you need the $state branch code to get this to work.

Anyway, here's the new syntax. Please let me know if you have any thoughts.

Syntax v2.0

Consists of :

• $state.set
• $state.get
• $state.reset

All of the use a key format that looks like this: [key path]@[url]

For example: username@https://www.jasonbase.com/things/3nf.json. This is similar to the partial mixin syntax, but a bit more restricted in that you can ONLY access the root keys.

This means you can't do things like user.username@https://www.jasonbase.com/things/3nf.json. Instead you should just fetch user@https://www.jasonbase.com/things/3nf.json and assign it to a local variable, and then use it.

Here are the details:

$state.set

Setting one attr.

{
  "type": "$state.set",
  "options": {
    "username@https://jasonbase.com/things/3nf.json": "ethan"
  }
}

Setting multiple attrs

{
  "type": "$state.set",
  "options": {
    "username@https://jasonbase.com/things/3nf.json": "ethan",
    "email@https://jasonbase.com/things/3nf.json": "ethan.gliechtenstein@gmail.com"
  }
}

$state.get

Getting one attr. In this case, it retrieves the stored value at username@https://jasonbase.con/cnf.json and then passes it to the succeeding action as $jason.username.

{
  "type": "$state.get",
  "options": {
    "username": "username@https://jasonbase.com/cnf.json"
  }
}

Getting multiple attrs. Like above, it fetches the value and passes them down to the succeeding action as {"$jason": {"username": "...", "email": "..."}}.

{
  "type": "$state.get",
  "options": {
    "username": "username@https://jasonbase.com/cnf.json",
    "email": "email@https://jasonbas.com/chnf.json"
  }
}

$state.reset

Resetting a single attr. Notice the options takes an array.

{
  "type": "$state.reset",
  "options": [ "username@https://jasonbase.com/things/3nf.json" ]
}

Resetting multiple attrs

{
  "type": "$state.reset",
  "options": [ "username@https://jasonbase.com/things/3nf.json", "email@https://jasonbase.com/things/33n.json" ]
}

[gliechtenstein]

Syntax v3.0

OK today I have another proposal. The v2.0 definitely works but I wasn't completely satisfied (especially the $state.get part) so have been trying to come up with a couple more ideas.

So this time I tried to come up with a syntax that will:

1. Get rid of the email-address-like scheme to refer to various states
2. Use a variable instead of an action when retrieving state

Anyway, here it goes:

A. Setting state

In this version, $state.set uses a full hierarchy instead of using the email address-like syntax (for example username@https://jasonbase.com/things/3nf.json)

1. A single attribute for a single source

{
  "type": "$state.set",
  "options": {
    "https://jasonbase.com/things/3nf.json": {
      "username": "ethan"
    }
  }
}

2. Multiple attributes from a single source

{
  "type": "$state.set",
  "options": {
    "https://jasonbase.com/things/3nf.json": {
      "username": "ethan",
      "email": "ethan.gliechtenstein@gmail.com"
    }
  }
}

3. Multiple attributes from multiple sources

{
  "type": "$state.set",
  "options": {
    "https://jasonbase.com/things/3nf.json": {
      "username": "ethan",
      "email": "ethan.gliechtenstein@gmail.com"
    },
    "https://jasonbase.com/things/333.json": {
      "lastUpdated": "{{Date.now().toString()}}"
    }
  }
}

B. Getting state

One thing I didn't like about previous approaches was you have to manually call $state.get everytime you need to read some state. This became quite verbose when I tried using it in a real world app, so I decided to try the approach used by cache and local variables--instead of using an action, directly access them using a special variable:

{
  "type": "label",
  "text": "{{$state['https://jasonbase.com/things/cnf.json'].username}}"
}

If the URL feels too bulky we can always move this to $load and initialize them into local variables, like this:

{
  "$jason": {
    "head": {
      "actions": {
        "$load": {
          "type": "$set",
          "options": {
            "db": "{{$state['https://jasonbase.com/things/cnf.json']}}"
          },
          "success": {
            "type": "$render"
          }
        }
      },
      "templates": {
        "body": {
          "sections": [{
            "items": [{
              "type": "label",
              "text": "{{$get.db.username}}"
            }]
          }]
        }
      }
    }
  }
}

C. Resetting State

1. Resetting a single attribute from a single source

{
  "type": "$state.reset",
  "options": {
    "https://jasonbase.com/things/33n.json": ["email"]
  }
}

2. Resetting multiple attributes from a single source

{
  "type": "$state.reset",
  "options": {
    "https://jasonbase.com/things/3nf.json": ["username", "email"]
  }
}

3. Resetting multiple attributes from multiple sources

{
  "type": "$state.reset",
  "options": {
    "https://jasonbase.com/things/3nf.json": ["username", "email"],
    "https://jasonbase.com/things/33n.json": ["lastUpdated"]
  }
}

If anyone sees something weird, please point out. Otherwise i'm gonna try to write an implementation for this in the next couple of days and test it out. Thanks

[moses5407]

I'm sure I'm missing a LOT .. but why not json data into and out of indexeddb something like this?
https://stackoverflow.com/questions/31703419/how-to-import-json-file-into-indexeddb

[gliechtenstein]

@moses5407 the problem is more about "how" to express the get/set/reset actions in pure JSON markup, than how to implement it internally.

The implementation itself is easy like you said, it will be pretty much about storing JSON under the URL namespace. What's been difficult was coming up with a concise and consistent way of expressing these actions in pure JSON.

Or maybe I misunderstood what you said? Could you clarify if I got your idea wrong? Thanks!

[moses5407]

You didn't misunderstand. :-)

In my simple mind, I was envisioning simply iterating over any changed data set and updating the indexeddb datastore or get-ing data from the indexeddb datastore in an offline condition.

I'm slowwwwwly trying to get up to speed on a number of things and offline first is only one of them.
In the process of looking into indexeddb I came across things like the following and it seemed pretty straightforward. Probably, my ignorance is (unrealistic) bliss. Hahaha!

var objstore = db.transaction([STORE], "readwrite").objectStore(STORE);
for (i = 0; i < data.length; i++) {
objstore.put(data[i]);
}

[drwasho]

Hey @gliechtenstein

1. Setting state

• Makes a great deal of sense and I like it

2. Getting state

so I decided to try the approach used by cache and local variables--instead of using an action, directly access them using a special variable

👏 Masterpiece

3. Reseting state

Makes sense. LGTM.

[hoffmabc]

@gliechtenstein could you just use "global" as your namespace for the cache and write it from wherever? I would assume this latest model allows that?

[gliechtenstein]

@hoffmabc it's for security reasons. Jasonette is basically a browser for apps. This means it is possible to cross over between different apps (the soul) within the same app (the shell). I think this is one of the biggest strengths of Jasonette, but at the same time can be a security hole. I think it's easy to understand if you think in parallel with how web browsers work. They have localstorage but each localstorage is sandboxed to the parent URL.

Here's a hypothetical scenario where it could go wrong if we didn't have the sandbox model:

1. App 1 stores some data in the global storage
2. App 1 opens App 2 via href
3. App 2 now has access to the global storage set by App 1.

App 1 may or may not have intended it, but the point is that there is no way to restrict this if we keep a single global storage.

The idea was that, by tying these with URL it's much easier to handle permissions (For example App 1 can decide to allow access to its own "global" storage to only a selected set of URLs)

This can be a powerful feature for an app, because you can allow access to the master localstorage from child views (only if you want), as well as specify strict permission policy between views, allowing for a more decentralized and customized implementation of apps.

Hope this makes sense. If you have any suggestions or different ideas please share.

[gliechtenstein]

OK I change my mind.

I have decided to roll out the "global variable" everyone wants, instead of trying to keep trying to come up with the best solution for a sandboxed storage. It's much simpler than everything I've mentioned above because it's just a single global scope, and there is no URL.

Example syntax:

Setting:

{
  "type": "$global.set",
  "options": {
    "db": ["a", "b", "c", "d"]
  }
}

Then you can use the db variable from anywhere in the app using:

{
  "items": {
    "{{#each $global.db}}": {
      "type": "label",
      "text": "{{this}}"
    }
  }
}

You can also remove the key from the global namespace:

{
  "type": "$global.reset",
  "options": {
    "items": ["db"]
   }
 }

The reason I changed my mind is:

1. I've talked to quite a few users about this feature, and from the feedback, the only thing people want for now is a dead simple way to store stuff globally so they can exist across views.
2. I do want to implement the sandboxed local storage because global variables are bad and the sandboxed model will be much more powerful due to its privacy and security model, BUT for now there are too many things to think about.
3. So I will release the global storage as a "true global storage". And hopefully figure out the best way to implement the sandboxed storage based on its usage.

The biggest benefit of this unrestricted global variable approach is that it's as simple as it can get, which is what everyone would want I presume.

Anyway here's a simple TODO app using the global variable where the parent view opens a child view as a modal where it adds an item and returns back to the parent.

I think this feature will make $util.picker and a lot of other widgets unnecessary because you can simply present a custom view to use as input instead of using these widgets, which was one of the most frequently asked questions--how to customize pickers and built-in widgets

Anyway, check out the branch: https://github.com/Jasonette/JASONETTE-iOS/tree/global

The code is annotated to show you how to use them (Jasonette/JASONETTE-iOS@525bbe9) and you can also refer to the example I shared above.

I plan to merge this soon, because this is super simple and there's not much that can go wrong with this. But if anyone has feedback on anything such as the naming (currently it uses $global but if you have a better idea let me know) or anything else, please let me know before I finalize.

Thanks!

[moses5407]

NOW I understand the reason for the sandboxed storage. Thanks for that explanation about APP1 calling APP2 and the possible resulting data confusion.

[j2l]

I also get the reasons behind your initial setup. Thanks