Update master (#2)

* Fixed marklogic-community#288: by-pass auth entirely in node with few lines

* marklogic-community#288: refined logic a bit

* Fixed permissions, improved init-local

* marklogic-community#288: refinements, and test issues

* Final changes

* Fixed a few missing dependencies

* Found some last minute bugs

* Some final refinements on the node-server

* Fixed marklogic-community#287: added helmet package for slightly better security

* Refined marklogic-community#287 based on latest HelmetJS changes

* Use an auth helper that can handle digest authentication marklogic-community#284

* Added pm2 config generation

* Updated README with how to use pm2

* Updated readme, tidied js files and replaced previous gulp tasks with gulp add-deploy-target

* Updated to support local.ecosystem.json and to have a separate Install guide

* adjusted code style

* Created an encoding variable in the slushfile

* Use passport LocalStrategy

* Add back lost logic on guest access, etc.

* Put skipBinary into it's own variable

* Updated readmes

* added feature to edit documents
added feature to delete documents
added feature to choose file type to save the data
added feature to validate data before saving

* default view of the document is based on the document format e.g. if the document is json, default view is json

* test

* fixed docFormat typo

* fix on conflicts

* fixed based on geert's review points

* ordered alphabetically the dependencies

* Implemented fixes based on review points

* Added delete confirmation

* change equality operations

* Fixed errors in unit tests

* Fix: Jquery should be listed ahead of any angular dependency to make sure angular uses it

* Fix for the map unit tests

* Fixes based on review points

* Fixed: map create form shows validation errors on open

* Added sudo and amended some of the commands for removing services

* Updated readme and removed /etc

* updated sample data to include docFormat

* Added form validations
Fixed unit test to support both create and edit

* Fixed confirmation message

* Moved the map detail buttons to the top
Moved the default details buttons to the right on top of similar panel

* justify buttons

* Add triples to sample data

This commit adds the following triples to each sample document:

<docUri> <rdfs:label> <doc/name> .
<docUri> <rdf:type> <foaf:Person> .

and three triples pointing to 3 random documents:

<docUri> <foaf:knows> <someRandomDocUriFromThisSampleSet>

There is also a single additional triple in the first doc
('data-0.json') that establishes a label for `foaf:knows`. This would
normally go into an ontology, but it seemed easiest just to embed it
into a document rather than introduce that much complexity. This
additional triple is needed for a nice OOTB display when using
[ml-visjs-graph-ng](https://github.com/patrickmcelwee/ml-visjs-graph-ng)

These triples were inserted into the original sample docs using [this
sample-triple-generator
library](https://github.com/patrickmcelwee/sample-triple-generator), in
case that is a helpful starting point for any future additions.

* Hide 'Edit' and 'Delete' when `disallowUpdates: true`

Previously, the buttons still appeared, but resulted in a blank page for
'Edit' and an unhandled server error for 'Delete'.

This requires added the userService and $scope as additional
dependencies in the generated detail controller.

It seems that only the map theme has an override for the detail
controller. I made the changes there as well.

* fixing the footer logo for bug marklogic-community#382

* Fixed marklogic-community#391: added cmd options for all questions

* Fixed marklogic-community#389: issue with gulp build and postcss for older node versions

* marklogic-community#391: allow running slushfile with gulp directly for easier testing

* marklogic-community#391: Allow app-name as gulp arg

* marklogic-community#391: ignore gulpfile arg

* marklogic-community#391: small bug in xcc-port

* marklogic-community#391: theme argument was not handled correctly

* Fixed marklogic-community#398: added OOTB logic for all Google overlay types

* Fixed marklogic-community#398: commented out marker/polyline, we don't do proximity search (yet)

* Define variables that were left undefined

Fixes marklogic-community#405

* for review

TODO: there is a bug in the co-occerence

* set no limit to get all combinations

* support broke on the new version of ml-highcharts-ng

* remove range element index on eyeColor in favor of the range path index

* use range path index in favor of the json index

* Fine-tuned dashboard charts, and fixed node-server proxy

* Fixed marklogic-community#397: proper init of auth-helper, and defaulting of user/pass

* marklogic-community#243: Initial progress with code cleanup and code coverage

* marklogic-community#243: minor bug in create ctrl tests

* marklogic-community#243: fixed gulp build for dashboard/map themes, further code and test coverage improvements, and code cleanup

* Fixed marklogic-community#384: setting unlimited timeout, ML will eventually time out

* Fixed marklogic-community#329: improved user endpoints in middle-tier

* Fixed marklogic-community#377: added stub user profile for admin user

* marklogic-community#384: capturing socket timeout for better exception handling

* Refined documentation and next step links, fixes marklogic-community#375, fixes marklogic-community#388, fixes marklogic-community#419, and fixes marklogic-community#421

* Fixed marklogic-community#385: replaced undocumented xdmp:content-type with official xdmp:uri-content-type

* Fixes marklogic-community#373: added extractedToJson.sjs rest transform

* Improving marklogic-community#291, and marklogic-community#410: added notes on mlcp-home to README

* Fixed link to INSTALL docs

* Fixed link to INSTALL docs

* Adding documentation for the proper use of MLCP in Windows.

* Fixed landing page panel issue, cleaned up rest transforms

* Release 1.2.0

* Release 1.2.0

* first commit for the marklogic-community#411 ticket. Missing the all.xml and ml-config.xml.

* fixing bug with the panels on the landing page

* Rebased cards-theme on top of latest changes, and consolidation across themes

* Fixed col-sx/xs typo

* Fixed typo in sanitize.xsl

* Added view-file-ng to detail page

* Test scripts

* Added showHome toggle

* Fixed app.route dependency

* Fixed app.route dependency

* Fixed app.route dependency

* refinements for marklogic-community#411. Added the blue variant

* changed the -xs- to 12 for repsonsive fix

* Fixed marklogic-community#433: temp fix for old node versions

* first commit for the marklogic-community#411 ticket. Missing the all.xml and ml-config.xml.

* fixing bug with the panels on the landing page

* Rebased cards-theme on top of latest changes, and consolidation across themes

* Fixed col-sx/xs typo

* Fixed typo in sanitize.xsl

* Added view-file-ng to detail page

* Test scripts

* Added showHome toggle

* Fixed app.route dependency

* Fixed app.route dependency

* Fixed app.route dependency

* refinements for marklogic-community#411. Added the blue variant

* changed the -xs- to 12 for repsonsive fix

* Do not add padding to .row in the dashboard theme

In Bootstrap, .row has no padding (and in fact has a negative margin)
to allow for nested columns. I just ran into this setting on a
dashboard-style demo, where I could not figure out why my nested columns
keep getting more and more indented.

I believe that this line of css was added to fix the search screen for
the dashboard theme. But a better fix is to use the "content" class
provided by AdminLTE. This is what I have done on my own demo, and it is
working well. It is also what the AdminLTE site does (though it is oddly
undocumented): https://almsaeedstudio.com/themes/AdminLTE/index2.html

* updating the colors to match the brand removal of blue. fixing a couple coloring issues as well

* Various changes for cards-theme

* adding fixes to the spacing for a fixed header

* make SSL support configurable

* rearranged notes

* change headings

* restore bower.json

* removed redundant log
fixed comment

*  rebase 401 work on cards-theme branch

* rearranged notes

* change headings

* removed redundant log
fixed comment

* Fixed typos in rest transforms

* SW-9  Added documentation on how to run in production.  This update creates a link to the documents.

* Fixed marklogic-community#442: making sure middle-tier uses digest auth against MarkLogic

* Added logic to error interceptor to redirect to a login screen when the user becomes unauthorised and formatted the create controller code

* adds new, explicitly white-listed REST API proxy

* Fixed marklogic-community#450: work-around for ml info throwing error + extra feedback about npm install, bower install, and gulp init

* Fixed marklogic-community#436: using os-homedir instead of ~

* Fixed marklogic-community#387: added note on how to accept RSA fingerprints + general cleanup of INSTALL.md

* Some more changes for marklogic-community#450, small bug with admin pwd handling in wizard, adjusted test script

* Fixed marklogic-community#451: index should point to min js/css, not .map, fixed. Also fixed endless gulp build, related to earlier commits for marklogic-community#450.

* Disabling term-options to allow ML to toggle options automatically

* Fixing typo in sanitize.xsl, again..

* Fixing content-db defaults in ml-config using xmlpoke, for better consistency accross ML versions

* Fixed marklogic-community#458: properly coded registered dateTime in sample-data

* Fixed marklogic-community#461: different handling of showHome false to fix navigation

* Improved fix for marklogic-community#438: skip vkbeautify-wrapper entirely

* Fixing gulp build issue

* Fix marklogic-community#467 - Allow /v1/values to proxy properly

* Updated CHANGELOG with current status

* Fixed marklogic-community#475: removed hardcoded sinon release

* Fixed marklogic-community#471: replaced @sample-app-role with dynamic -role in mlcp options

* Fixed marklogic-community#470: fixed handling of appname in login service

* Fixed marklogic-community#465: renamed ctrl to  in all html views

* Fixed  in directives

* Fixed marklogic-community#482: made sure authenticators are cleared properly at /user/logout, and code reformatting of all node-server files

* getAuthenticatedStatus needs to always return promise

If the _isAuthenticated memo was already set, it was returned a bald
value, without wrapping in a promise

Also, I found and threw the error that wasn't showing up - which would
have helped debug this issue.

Fixes marklogic-community#480

* Simplify consumer of getAuthenticatedService

Does not need to check if it returns promise or not, since always should

* Wrap getUser return to always return promise

* Included WINDOWS readme to generated apps

* Final touches on READMEs

* Making sure extsimilar works on binaries

* Fixed marklogic-community#484: no longer extract full doc at search by default

* Fixed small issue with Cancel on Create/Edit page

* Updated CHANGELOG and package.json for 1.3.0

* fixes for marklogic-community#490 which is to remove the blue less files

* Updated CHANGELOG

* Updated favicon

.gitignore

@@ -1,3 +1,10 @@
 node_modules
 .DS_Store
 /tmp/
+/ml
+/ml.bat
+/slush-default
+/slush-3column
+/slush-dashboard
+/slush-map
+/slush-cards

README.mdown

@@ -1,21 +1,20 @@
 # Generator of Node.js Application Backed by MarkLogic
 
-This is a [Slush](http://slushjs.github.io/) generator that produces a skeleton
-[MarkLogic](http://www.marklogic.com/) application with the following
-components:
+This is a [Slush](http://slushjs.github.io/) generator template that
+produces a skeleton [MarkLogic](http://www.marklogic.com/) application
+with the following components:
 
 - [AngularJS](https://angularjs.org/)
 - [Gulp](http://gulpjs.com/)
 - [node.js](http://nodejs.org/): very thin layer, hosting the Angular code and proxying MarkLogic REST API requests
 - [Roxy Deployer](https://github.com/marklogic/roxy): bootstrap MarkLogic databases, application servers, etc; scaffolding for MarkLogic REST API service extensions
 
-# Getting Started
-
-Microsoft Windows users should read the [additional requirements documentation](WINDOWS.mdown)
 ## Install Required Dependencies
 
-- [node.js](http://nodejs.org/download/)
-- [npm](https://www.npmjs.com/): Built-in package manager for node (comes with
+To run this generator, you need:
+
+- [node.js v0.10+](http://nodejs.org/download/)
+- [npm v2+](https://www.npmjs.com/): Built-in package manager for node (comes with
   node, but check to be sure you have latest version: `npm install -g npm`)
 - [gulp](http://gulpjs.com/): Javascript task automation (`npm install -g
   gulp`)
@@ -24,110 +23,175 @@ Microsoft Windows users should read the [additional requirements documentation](
 - [Bower](http://bower.io/): A package manager for front-end libraries (`npm
   install -g bower`)
 - [Git](https://git-scm.com/) - Roxy depends on this version control system
-- [Ruby](https://www.ruby-lang.org/en/documentation/installation/) - Roxy
+- [Ruby v1.9.3+](https://www.ruby-lang.org/en/documentation/installation/) - Roxy
   depends on Ruby in order to run server configuration scripts
 
-## Install the generator
+Note: the `node` command has been renamed to `nodejs`. Some dependencies still point to `node`, which is out of our control.
+You may need to manually alias those commands, or install the `node-legacy` package.
+
+Microsoft Windows users should also read the [Additional Microsoft Windows Requirements](WINDOWS.mdown).
+
+## Install the Generator
+
+The recommended way to install this generator is:
 
     npm install -g slush-marklogic-node
 
-If you prefer to use the latest code under development:
+To grab latest features under development you can use:
 
     npm install -g git://github.com/marklogic/slush-marklogic-node.git
 
-## Run the generator
+## Run the Generator
 
-Go to the directory where your new project will be created (the parent). Then:
+Go to the directory under which your new project should be created (the parent), and run:
+
+    slush marklogic-node
+
+Or:
 
     slush marklogic-node <app-name>
 
-This will ask you several questions, with some intelligent defaults.
+This will ask you several questions, with some intelligent defaults. The end result should
+be a subdirectory with a name matching the app-name, containing all the application code.
+
+## Quick Start (for the Impatient)
+
+On Mac or Linux:
 
-## Configure your application
+    cd <app-name>
+    npm install
+    bower install
+    gulp init-local
+    ./ml local bootstrap
+    ./ml local deploy modules
+    ./ml local deploy content
+    ./ml local mlcp -options_file import-sample-data.options
+
+On Windows:
+
+    cd <app-name>
+    npm install
+    bower install
+    gulp init-local
+    ml.bat local bootstrap
+    ml.bat local deploy modules
+    ml.bat local deploy content
+    ml.bat local mlcp -options_file import-sample-data.options
+
+## Prepare your Application
 
 Go to the generated application directory:
 
-    cd {app-name}
+    cd <app-name>
+
+The generator installs NodeJS and JavaScript dependencies automatically,
+but if shared among colleagues, they likely need to run the following commands
+manually:
+
+    npm install
+    bower install
+
+The generator also creates a `local.json` and a `deploy/local.properties`, but
+those are usually gitignored. Check if they exist. If they do, check the settings,
+most importantly the port settings to make sure they list available ports. If
+those files don't exist, then use the following command to create them:
+
+    gulp init-local
+
+Note: you can use `gulp init-dev` and `gulp init-prod` to setup properties for
+the dev and prod environments.
+
+Note: the deploy properties allow tweaking the MarkLogic side of the application
+in great detail. Look inside `deploy/build.properties` or the documentation of
+the [Roxy deployer](https://github.com/marklogic/roxy) to learn more about this.
 
-Edit `deploy/build.properties` to set the -port properties to available ports
-(defaults: 8040, 8041) and change other properties. The file itself documents
-many of those properties. Others are described in the [documentation for the
-Roxy deployer](https://github.com/marklogic/roxy). Then run the following Roxy
-commands to configure your local MarkLogic database (typically, depending on
-your configuration, it will create a database, associated forest(s), a REST
-server, an app-user, and a role for that user).
+Note: consider editing the `_loginMode` variable in `ui/app/login/login.service.js`
+to change the login mode of the application. The default is set to `full`. The
+other options are `top-right` or `modal`.
+
+## Deploy your Application
+
+Run the following Roxy commands to deploy the application to the chosen MarkLogic
+installation. It will create and configure databases, REST servers, users, and
+roles for you, and deploy the back-end application code.
 
     ./ml local bootstrap
     ./ml local deploy modules
+    ./ml local deploy content
 
-On Windows, that would be:
+Or on Windows:
 
     ml.bat local bootstrap
     ml.bat local deploy modules
+    ml.bat local deploy content
 
-Install additional dependencies using the bower package manager:
+## Launch your Application
 
-    bower install
+To launch the middle-tier, and open a browser tab automatically, you only need to run:
 
-Edit the `_loginMode` variable in `ui/app/login/login.service.js` to change the login mode of the application. The default is set to `full`. The other options are `top-right` or `modal`.
+    gulp serve-local
 
-## Run your application
+Note: this will also watch the `ui/` folder for changes, compile them, and reload the
+browser fully automatically.
 
-    gulp serve-local # this will watch the .less file for changes, compile them to .css, and run the node server
+Note: to get more debug logging from Express, run with:
 
-You can set some settings to match your setup in the Roxy properties files. For example:
+    DEBUG=http,mail,express:* gulp serve-local
 
-    gulp serve-local --ml-host=<your-host> --ml-http-port=8234 --app-port=9234 --nosync
+Note: it is possible to override settings on the command-line:
 
-# Next Steps
+    gulp serve-local --ml-host=<your-host> --ml-http-port=8234 --app-port=9234 --nosync
 
-Load some sample data, as described in the next section of this README.
+Note: `--nosync` will cause no extra browsersync reload proxy to get launched at 3000,
+and no browser tab will be opened automatically. In addition you have `--verbose`, and
+`--ignoreErrors` (for `gulp build`). A few more flags are documented at the top of
+`gulpfile.js`.
 
-The slush generation creates an entire project directory, described in
-[Project folder structure](https://github.com/marklogic/slush-marklogic-node/wiki/Project-folder-structure).
+## Loading Sample Data (optional)
 
-[Explaining the
-stack](https://github.com/marklogic/slush-marklogic-node/wiki/Explaining-the-stack)
-gives a more complete description of the overall architecture.
+The application comes with 3000 JSON documents generated by json-generator.com. They
+will allow you to explore all the features you get out of the box in a better way. You
+can load them with (MLCP)[https://docs.marklogic.com/guide/ingestion/content-pump] using
+Roxy.
 
-Learn more about specific components of the generated application on the
-[Components](https://github.com/marklogic/slush-marklogic-node/wiki/Components) page.
+Before you hit off, check if Roxy can find MLCP:
 
-Learn about specific ways of customizing your app on the [Customization
-Recipes](https://github.com/marklogic/slush-marklogic-node/wiki/Customization-recipes)
-page.
+    ./ml local mlcp
 
-# Data
+Roxy will print the following message if it cannot find MLCP:
 
-## Sample Data
+    ERROR: MLCP not found or mis-configured, please check the mlcp-home setting.
 
-The application comes with 3000 JSON documents generated by json-generator.com.
-You can load them with
-(MLCP)[https://docs.marklogic.com/guide/ingestion/content-pump] like this:
+Roxy looks for `/usr/local/mlcp/` (or `\usr\local\mlcp\`) by default. It can be
+convenient to just install MLCP there, or create a symlink from `/usr/local/mlcp`
+to where MLCP is installed. Alternatively, edit your `deploy/local.properties`, and
+append a `mlcp-home=/path/to/your/mlcp/` to the end of it. Repeating above command
+should show MLCP usage, not an ERROR.
 
-**NOTE:** @sample-app-role in import-sample-data.options will change to your app's role
-(Alternatively, you can discover that with `./ml local info`)
+Once MLCP works correctly, you can run:
 
     ./ml local mlcp -options_file import-sample-data.options
 
 Or on Windows:
 
     ml.bat local mlcp -options_file import-sample-data.options
 
-## General Data Information
+Note: the detail controller, the part that handles showing your data, can not only
+handle JSON, but also XML, Binary, and Text data out of the box.
+
+## Deployment on Server
+
+At some point you might want to deploy and run this as a service on some
+(demo) server. For details on that see [INSTALL.mdown](app/templates/INSTALL.mdown).
 
-The application assumes that you're storing JSON data. This shows up in the
-default format request for the MLRest service's `searchContext`, the
-`detailController`'s (`detail-ctrl.js`) request to get a document, and in the
-out-of-the-box detail view.
+## Next Steps
 
-# Options
+Documentation about the slush generator is provided on the [Generator Wiki](https://github.com/marklogic/slush-marklogic-node/wiki). It covers amongst others:
 
-## Roxy
+- [Project folder structure](https://github.com/marklogic/slush-marklogic-node/wiki/Project-folder-structure), which describes the project directory as produces by the generator.
 
-The generator defaults to using a 'rest'-type Roxy app and the 'master' branch
-of Roxy. If you know what you're doing, and you want to do something different,
-you can specify those options with this syntax:
+- [Explaining the stack](https://github.com/marklogic/slush-marklogic-node/wiki/Explaining-the-stack), which gives a brief description of the overall architecture.
 
-    slush marklogic-node <appName> appType=mvc branch=dev
+- [Core Tools and Components](https://github.com/marklogic/slush-marklogic-node/wiki/Core-Tools-and-Components), which allows you to learn more about the various tools and components used by or for the application.
 
+Last but not least, the landing page of the out of the box application provides many more links.

WINDOWS.mdown

@@ -1,4 +1,4 @@
-# Additional Microsoft Windows Requirement
+# Additional Microsoft Windows Requirements
 
 Some required modules do not have native Windows modules available and require a development environment in which they can be compiled. To do this, npm uses node-gyp to create these modules.  For more information, please read the [node-gyp documentation](https://github.com/nodejs/node-gyp/blob/master/README.md)
 
@@ -12,3 +12,11 @@ Some required modules do not have native Windows modules available and require a
 ## Use of Elevated Command Prompt
 
 If your installation of node and npm required elevated priviledges, you will need to use an elevated command prompt to upgrade npm and perform all global `npm install -g` commands. When scaffolding your project using slush, a normal command prompt is sufficient.
+
+## Configuring MLCP
+
+The default location for MLCP is `/usr/local/mlcp`.  This path is unlikely the appropriate location of your MLCP install.  To specify the correct location of MLCP in Windows, edit/create `deploy\local.properties` and add
+```
+mlcp-home=C:\\path\\to\\mlcp
+```
+Please note that the backslash is being escaped by a second backslash.  This inserts a literal backslash instead of inserting an escape character in Java.