Streaming: refactor streaming into adapter. General improvements and quality.

[carlosscastro]

Fixes #1178

Description

The high level purposes of this change are mainly:

1. Integrate multiple streaming transports (web sockets and named pipes) into the BotFramework adapter
2. All code that worked before, works the same way now. Named pipes and web sockets are opt in only and the new code paths should only trigger as response to requests through those transports.
3. General rearrangement of libraries. Removing botbuilder-streaming-extensions, moving content into the bot framework adapter, and renaming botframework-streaming-extensions -> botframework-streaming
4. General quality improvements: Code comments, documentation and error management improvement.

Usage

Usage: Http only bot

server.post('/api/messages', (req, res) => {
    adapter.processActivity(req, res, async (context) => {
        // Route to bot
        await myBot.run(context);
    });
});

Usage: WebSockets bot

server.get('/api/messages', (req, res) => {
    adapter.processActivity(req, res, async (context) => {
        // Route to bot
        await myBot.run(context);
    });
});

Specific Changes

1. Move logic BotFrameworkStreamingAdapter into BotFrameworkAdapter
2. Expose useWebSocket and useNamedPipe methods to enable socket and named pipes respectively
3. Delete botbuilder-streaming-extension project
4. Rename botframework-streaming-extensions -> botframework-streaming
5. Code comments: Move from CSharp style comments to javascript style comments
6. Code comments: Add documentation to undocumented classes and methods that are part of the surface
7. Update tests to new structure
8. Directory and file casing: Rename directories and files to be consistent with the naming convention of the project
9. Remove double quotes
10. Logging: remove console logging in the adapter and use the http response body to complement the status code.
11. Error messages: Include actionable error information in messages for easier debugging
12. Decouple streaming adapter from bot

Testing

We have 4 sources of testing

1. Streaming extensions: This code has been available as part of the streaming extensions preview package and we've had feedback and bugs reported, which were promptly fixed.
2. [In progress] E2E directline tests: Once this gets merged and directline js consumes it, we'll set up end to end tests, but they will run externally as part of the directline pipeline
3. [In progress] Functional tests: @DDEfromOR and I are figuring out how to run functional tests as part of the build.
4. Unit tests: Thorough code coverage in the checked in code. Report below:

Code Coverage Report[1]

[1] Source

[coveralls]

Pull Request Test Coverage Report for Build 84085

• 215 of 242 (88.84%) changed or added relevant lines in 31 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.3%) to 87.621%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libraries/botframework-streaming/src/webSocket/webSocketClient.ts	2	3	66.67%
libraries/botbuilder/src/streamingHttpClient.ts	9	12	75.0%
libraries/botframework-streaming/src/webSocket/browserWebSocket.ts	0	4	0.0%
libraries/botbuilder/src/botFrameworkAdapter.ts	89	108	82.41%

Totals
Change from base Build 83819:	-0.3%
Covered Lines:	4849
Relevant Lines:	5324

---

💛 - Coveralls

[DDEfromOR]

I'm a little confused why Master has v4.1.6 for all packages, are they really that far out of date or is there something special about the value?
Technically we only support Streaming for BotBuilder 4.5 or higher, but I've tested with the 4.1.6 labeled packages in Master and everything works.

[carlosscastro]

I just stayed consistent with the rest of packages for now. What version should we ref? ^4.5?

[DDEfromOR]

It will work with 4.1.6, we just don't technically support it. I think our options are:
1 - Update the rest of the packages to the current release versions.
2 - Go with 4.1.6 for consistency.
3 - Go with ^4.5 with the knowledge it creates a bad experience for developers trying to build off of master.

[DDEfromOR]

Do we need type identifiers here? :string

[DDEfromOR]

Awesome work! It feels great to see the adapter merged down and the interim package removed. Thank you for getting this over the finish line, Carlos!

[carlosscastro]

Reentrancy

[carlosscastro]

consider warning + override

[carlosscastro]

Cardinality, simulateneous socket + named pipe

[carlosscastro]

consider map based

[carlosscastro]

virtual accept for people t manage their own socket

[stevengum]

Initial feedback