Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/es support server #291

Merged
merged 35 commits into from
Apr 27, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
b2457b9
refactoring server
just-at-uber Apr 22, 2021
aec6f41
more refactoring
just-at-uber Apr 23, 2021
896cb5a
fix lint
just-at-uber Apr 23, 2021
4e8de81
moved router into its own folder. moved helpers into router folder. s…
just-at-uber Apr 23, 2021
12989de
Merge branch 'master' into feature/es-support
just-at-uber Apr 23, 2021
6952a2e
fix conflicts
just-at-uber Apr 23, 2021
c17d10a
moving code into separate files for route handlers
just-at-uber Apr 23, 2021
366f3a5
fix lint
just-at-uber Apr 23, 2021
91ce070
Merge branch 'master' into feature/es-support
just-at-uber Apr 23, 2021
dff0856
adding cluster handler
just-at-uber Apr 23, 2021
59dd932
fix lint
just-at-uber Apr 23, 2021
5117b29
piped through tchannel changes to support admin service requests. com…
just-at-uber Apr 23, 2021
9a80a84
adding caching to cluster handler
just-at-uber Apr 23, 2021
5447c1b
adding cluster cache ttl constant
just-at-uber Apr 23, 2021
ccd8f49
revert frontend changes
just-at-uber Apr 23, 2021
21eeb7e
fix lint
just-at-uber Apr 23, 2021
1412c2f
artificial change to test integration tests.
just-at-uber Apr 26, 2021
bdd718a
Merge branch 'master' into feature/es-support-server
just-at-uber Apr 26, 2021
9dbcd34
Merge branch 'master' into feature/es-support-server
just-at-uber Apr 27, 2021
6536ad7
revert readme
just-at-uber Apr 27, 2021
a1520d0
Merge branch 'feature/es-support-server' into fix/flakey-integration-…
just-at-uber Apr 27, 2021
a7f0b27
adding timeout
just-at-uber Apr 27, 2021
d2587f0
add done for async test
just-at-uber Apr 27, 2021
042ce11
reducing timeout to 500ms
just-at-uber Apr 27, 2021
c57c35e
fixed timeout as this is the timeout for the test before failing
just-at-uber Apr 27, 2021
73173aa
increasing timeout time
just-at-uber Apr 27, 2021
9e4321e
removing done function as it does not work
just-at-uber Apr 27, 2021
466d1ac
increase timeout to 20000ms
just-at-uber Apr 27, 2021
7548afd
increasing timeout to 50K ms
just-at-uber Apr 27, 2021
d9a1a9a
scrapping timeout and inserting delay to test
just-at-uber Apr 27, 2021
61f28b5
tests now passing. trying to reduce timeout for faster runtime
just-at-uber Apr 27, 2021
c6d59b9
increasing delay to 200ms
just-at-uber Apr 27, 2021
ea4b459
increasing delay to 300ms
just-at-uber Apr 27, 2021
b91ddf3
increasing to 500ms
just-at-uber Apr 27, 2021
153436d
increasing delay to 1000ms
just-at-uber Apr 27, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions client/test/workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,9 @@ describe('Workflow', () => {
);

terminateEl.trigger('click');

await Promise.delay(1000);

const confirmTerminateEl = await summaryEl.waitUntilExists(
'[data-modal="confirm-termination"]'
);
Expand Down
2 changes: 1 addition & 1 deletion server/middleware/tchannel-client/helpers/format-method.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

const formatMethod = method => `WorkflowService::${method}`;
const formatMethod = ({ method, serviceName }) => `${serviceName}::${method}`;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serviceName can now be passed in as an option to request. Defaults to WorkflowService.


module.exports = formatMethod;
4 changes: 2 additions & 2 deletions server/middleware/tchannel-client/helpers/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const formatBody = require('./format-body');
const formatMethod = require('./format-method');
const formatRequestName = require('./format-request-name');
const lookupAsync = require('./lookup-async');
const makeChannel = require('./make-channel');
const makeChannels = require('./make-channels');
const makeRequest = require('./make-request');
const uiTransform = require('./ui-transform');
const withDomainPagingAndWorkflowExecution = require('./with-domain-paging-and-workflow-execution');
Expand All @@ -38,7 +38,7 @@ module.exports = {
formatMethod,
formatRequestName,
lookupAsync,
makeChannel,
makeChannels,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeChannel renamed to makeChannels and now returns 2 channels.

makeRequest,
uiTransform,
withDomainPagingAndWorkflowExecution,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ const TChannelAsThrift = require('tchannel/as/thrift');
const { PEERS } = require('../constants');
const lookupAsync = require('./lookup-async');

const makeChannel = async client => {
const makeChannels = async client => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeChannel renamed to makeChannels and now returns 2 channels.

const ipPeers = await Promise.all(
PEERS.map(peer => {
const [host, port] = peer.split(':');
Expand All @@ -47,12 +47,20 @@ const makeChannel = async client => {
},
});

const tchannelAsThrift = TChannelAsThrift({
const adminTChannelAsThrift = TChannelAsThrift({
channel: cadenceChannel,
entryPoint: path.join(__dirname, '../../../idl/admin.thrift'),
});

const cadenceTChannelAsThrift = TChannelAsThrift({
channel: cadenceChannel,
entryPoint: path.join(__dirname, '../../../idl/cadence.thrift'),
});

return tchannelAsThrift;
return {
admin: adminTChannelAsThrift,
cadence: cadenceTChannelAsThrift,
};
};

module.exports = makeChannel;
module.exports = makeChannels;
10 changes: 6 additions & 4 deletions server/middleware/tchannel-client/helpers/make-request.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,18 @@ const formatMethod = require('./format-method');
const formatRequestName = require('./format-request-name');
const uiTransform = require('./ui-transform');

const makeRequest = ({ authTokenHeaders, channel, ctx }) => ({
const makeRequest = ({ authTokenHeaders, channels, ctx }) => ({

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we use the same auth token for the both channels?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good question. I need to be able to test this in staging to know whether we need different credentials. Right now I'm not sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be a massive change in the event it does need separate credentials.

bodyTransform,
channelName = 'cadence',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

channelName can now be passed in as an option to request. Defaults to cadence.

method,
requestName,
bodyTransform,
responseTransform,
serviceName = 'WorkflowService',
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serviceName can now be passed in as an option to request. Defaults to WorkflowService.

}) => body =>
new Promise((resolve, reject) => {
try {
channel.request(REQUEST_CONFIG).send(
formatMethod(method),
channels[channelName].request(REQUEST_CONFIG).send(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeChannel renamed to makeChannels and now returns 2 channels.

formatMethod({ method, serviceName }),
{
...authTokenHeaders,
},
Expand Down
12 changes: 9 additions & 3 deletions server/middleware/tchannel-client/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ const TChannel = require('tchannel');

const {
cliTransform,
makeChannel,
makeChannels,
makeRequest,
withDomainPaging,
withWorkflowExecution,
Expand All @@ -37,10 +37,10 @@ const tchannelClient = async function(ctx, next) {
const { authTokenHeaders = {} } = ctx;

const client = TChannel();
const channel = await makeChannel(client);
const channels = await makeChannels(client);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makeChannel renamed to makeChannels and now returns 2 channels.

const request = makeRequest({
authTokenHeaders,
channel,
channels,
ctx,
});

Expand All @@ -55,6 +55,12 @@ const tchannelClient = async function(ctx, next) {
requestName: 'list',
bodyTransform: withDomainPaging(ctx),
}),
describeCluster: request({
channelName: 'admin',
method: 'DescribeCluster',
requestName: 'describe',
serviceName: 'AdminService',
}),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clusterHandler to return cluster data from cadence server. Caches data for 1 hour.

describeDomain: request({
method: 'DescribeDomain',
requestName: 'describe',
Expand Down
27 changes: 27 additions & 0 deletions server/router/constants.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Copyright (c) 2021 Uber Technologies Inc.
//
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

const ONE_HOUR_IN_MILLISECONDS = 60 * 60 * 1000;
const CLUSTER_CACHE_TTL = ONE_HOUR_IN_MILLISECONDS;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caches data for 1 hour.


module.exports = {
CLUSTER_CACHE_TTL,
};
4 changes: 4 additions & 0 deletions server/router/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
const Router = require('koa-router');

const {
clusterHandler,
domainAuthorizationHandler,
domainHandler,
domainListHandler,
Expand All @@ -45,7 +46,10 @@ const { listWorkflows } = require('./helpers');

const router = new Router();

router.get('/api/cluster', clusterHandler);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clusterHandler to return cluster data from cadence server. Caches data for 1 hour.


router.get('/api/domains', domainListHandler);

router.get('/api/domains/:domain', domainHandler);

/**
Expand Down
41 changes: 41 additions & 0 deletions server/router/routes/cluster-handler.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// Copyright (c) 2021 Uber Technologies Inc.
//
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in
// all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clusterHandler to return cluster data from cadence server. Caches data for 1 hour.


const { CLUSTER_CACHE_TTL } = require('../constants');

let cache = null;

const clusterHandler = async ctx => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the handler exposed somewhere on UI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be eventually, right now this is only containing the server side code changes. will raise separate PR which hooks this up in the frontend.

if (cache) {
return (ctx.body = cache);
}

const cluster = await ctx.cadence.describeCluster();

cache = { ...cluster, membershipInfo: null };
ctx.body = cache;

// This timeout will clear cache after TTL period.
// It will fetch a new value on the next request to clusterHandler.
setTimeout(() => (cache = null), CLUSTER_CACHE_TTL);
};

module.exports = clusterHandler;
2 changes: 2 additions & 0 deletions server/router/routes/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.

const clusterHandler = require('./cluster-handler');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clusterHandler to return cluster data from cadence server. Caches data for 1 hour.

const domainAuthorizationHandler = require('./domain-authorization-handler');
const domainHandler = require('./domain-handler');
const domainListHandler = require('./domain-list-handler');
Expand All @@ -38,6 +39,7 @@ const workflowSignalHandler = require('./workflow-signal-handler');
const workflowTerminateHandler = require('./workflow-terminate-handler');

module.exports = {
clusterHandler,
domainAuthorizationHandler,
domainHandler,
domainListHandler,
Expand Down