ZEPPELIN-497 ] pyspark completion

[cloverhearts]

Added auto-complete for pyspark.
autocomplete function will operate through the spark python tag '% pyspark'.
short-cut = Ctrl + Shift + Space.

jira issue : https://issues.apache.org/jira/browse/ZEPPELIN-497

[felixcheung]

if text_value is None should it returned all objects/names in the current scope (dir())?

[elbamos]

@felixcheung

Miscellaneous requests for improvement belong in jiras.

Rather than giving everyone suggestions on what to do in their PRs, perhaps you should try making them as PRs of your own.

[felixcheung]

tested and working well! thanks for working on this

[prabhjyotsingh]

Tested, works well 👍

[cloverhearts]

Filtering for FunctionName added.
Thank you for Feed back.

[Leemoonsoo]

Working well. Thanks for very useful improvement.
LGTM, Merge if there're no more discussions.

[felixcheung]

@cloverhearts thanks - to clarify the rule is two leading underscores (__foo)
https://www.python.org/dev/peps/pep-0008/#method-names-and-instance-variables
"To avoid name clashes with subclasses, use two leading underscores to invoke Python's name mangling rules. Python mangles these names with the class name: if class Foo has an attribute named __a , it cannot be accessed by Foo.__a"

you do not match both leading and trailing (__foo__)

[cloverhearts]

Changed Filter Rule.

Filter __FunctionName

NAME The form is not a target filter.

[felixcheung]

sorry if it's confusing - the method name mangling rule in Python applies to any name that starts with __, so it includes both __foo and __foo__ (eg. __getattr__)

so I think the regex should be ^__.+ or ^__.*

[cloverhearts]

@felixcheung
Sorry, I just modified. Tested after upload.

[cloverhearts]

Filtering

NAME and __NAME

Tested.

[felixcheung]

looks good - thank you!

[cloverhearts]

@felixcheung Thank you for Feed back! have a nice day!

[Leemoonsoo]

LGTM
Merge if there're no more discussions

[fanlu]

version 0.7.0-SNAPSHOT
build by mvn clean package -Pbuild-distr -Pspark-2.0 -Dspark.version=2.0.0 -Pscala-2.11 -Phadoop-2.6 -Dhadoop.version=2.6.0-cdh5.7.0 -Pyarn -Psparkr -Ppyspark -DskipTests -Drat.skip=true -U
when I use ctrl+. the stacktrace is
ERROR [2016-09-26 22:02:25,834]({qtp532445947-596} NotebookServer.java[onMessage]:257) - Can't handle message
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:453)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.completion(LazyOpenInterpreter.java:122)
at org.apache.zeppelin.notebook.Paragraph.completion(Paragraph.java:242)
at org.apache.zeppelin.notebook.Note.completion(Note.java:520)
at org.apache.zeppelin.socket.NotebookServer.completion(NotebookServer.java:793)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:216)
at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:56)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:128)
at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:122)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:161)
at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:309)
at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:214)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:220)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:258)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:632)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:480)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_completion(RemoteInterpreterService.java:358)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.completion(RemoteInterpreterService.java:342)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:449)
... 21 more

[felixcheung]

hi @fanlu - it's best to report issue to dev@zeppelin.apache.org

[astroshim]

@fanlu As @felixcheung mentioned, you can report issue to dev@zeppelin.apache.org.
and Could you test #1465 for solving your problem?

[fanlu]

@astroshim
sorry, the problem is still there
#1465 (comment)