Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azurecr/registry management #383

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
05b3138
Merge pull request #21 from AzureCR/master
Jul 25, 2018
a7be0b3
Added Azure Credentials Manager Singleton (#18)
Jul 25, 2018
b2e5740
Sorted Existing Create Registry ready for code review
Jul 25, 2018
ef1ccfa
Added acquiring telemetry data for create registry
rsamai Jul 25, 2018
1fca0da
broke up createnewresourcegroup method and fixed client use
rsamai Jul 26, 2018
60f68d3
Jackson esteban/unified client nit Fix (#24)
Jul 26, 2018
26f00e2
Enabled location selection
rsamai Jul 26, 2018
fe0fe2f
Location request fixes
rsamai Jul 30, 2018
5c8d364
Refactor while loop for new res group
rsamai Jul 30, 2018
a61c728
Added SKU selection
rsamai Jul 30, 2018
9b51397
Quick fix- initializing array syntax
rsamai Jul 30, 2018
048eb67
Added specific error messages and comments
rsamai Aug 1, 2018
bc905d8
Merge pull request #30 from AzureCR/master
rsamai Aug 2, 2018
37d8a27
Merge pull request #22 from AzureCR/Esteban-Julia-Rutu/CreateRegistry
rsamai Aug 2, 2018
b80af87
Julia/delete image (#29)
julialieberman Aug 4, 2018
e2cf013
Merge branch 'dev' into master
jvstokes Aug 7, 2018
64c8282
Merge pull request #42 from AzureCR/master
jvstokes Aug 7, 2018
05df67c
Estebanreyl/dev merge fixes (#43)
Aug 7, 2018
126c01e
Rutusamai/list build tasks for each registry (#37)
rsamai Aug 7, 2018
0a3d6e1
Julia/delete repository final (#49)
Aug 9, 2018
19f8b72
Julia/delete registry final (#47)
Aug 9, 2018
06b0c99
Merge branch 'master2' into dev
Aug 13, 2018
d694255
Merge pull request #52 from AzureCR/master-dev-merge
rsamai Aug 13, 2018
4bd802b
began updating
Aug 13, 2018
3d38215
Reorganized create registry and delete azure image
Aug 14, 2018
ca723eb
continued improvements
Aug 14, 2018
3964731
Began updating login
Aug 14, 2018
41429c5
being credentials update
Aug 14, 2018
5d84452
further updates
Aug 14, 2018
da96bf0
Finished updating, need to test functionality now
Aug 14, 2018
7a35685
Updated requests, things all work now
Aug 15, 2018
85798ae
Applied some nit fixes
Aug 15, 2018
249839a
Updates to naming
Aug 15, 2018
9963f25
maintain UtilityManager standards
Aug 15, 2018
a8b0f82
Updated Prompts
Aug 15, 2018
47c1ae4
Updated imports and naming / standarized telemetry
Aug 16, 2018
b921791
Added explorer refresh capabilities on delete/add
Aug 16, 2018
042255f
Remove build task features from this branch
Aug 16, 2018
16edcf6
Merge bugfixes and name specification
Aug 16, 2018
1822adf
updated weird naming issue
Aug 16, 2018
b950564
Merge remote-tracking branch 'upstream/master' into azurecr/registryM…
Aug 22, 2018
8dcd3fe
Deleted deprecated telemetry, added copyright comment and updated qui…
Aug 22, 2018
cd95434
Updated casing
Aug 22, 2018
96518e7
Updated resource group and registry validity checking and other nit f…
Aug 23, 2018
24b4d09
Updated Azure Utility Manager to by default sort registries alphabeti…
Aug 23, 2018
014930a
Updated azureRegistryNodes and registryRootNode to use shared functions
Aug 23, 2018
dd2482c
Corrected resourcegroup name test
Aug 23, 2018
6c5fb36
added delete button when deleting an image
Aug 23, 2018
105a611
Small changes in variables for better prompts and success notifications
rsamai Aug 24, 2018
998f639
Resolve merge conflicts and update no azureAccount issue
Aug 25, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion commands/utils/quick-pick-azure.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import * as opn from 'opn';
import * as vscode from "vscode";
import { IAzureQuickPickItem, UserCancelledError } from 'vscode-azureextensionui';
import { skus } from '../../constants'
import { QuickPickItemWithData } from '../../explorer/deploy/wizard';
import { ext } from '../../extensionVariables';
import { ResourceManagementClient } from '../../node_modules/azure-arm-resource';
import * as acrTools from '../../utils/Azure/acrTools';
Expand Down
200 changes: 37 additions & 163 deletions explorer/models/azureRegistryNodes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@ import * as ContainerModels from 'azure-arm-containerregistry/lib/models';
import { SubscriptionModels } from 'azure-arm-resource';
import * as moment from 'moment';
import * as path from 'path';
import * as request from 'request-promise';
import * as vscode from 'vscode';
import { parseError } from 'vscode-azureextensionui';
import { MAX_CONCURRENT_REQUESTS } from '../../constants'
import { AzureAccount, AzureSession } from '../../typings/azure-account.api';
import { AzureAccount } from '../../typings/azure-account.api';
import { AsyncPool } from '../../utils/asyncpool';
import { acquireACRAccessTokenFromRegistry, getImagesByRepository, getRawImageManifest, getRepositoriesByRegistry } from '../../utils/Azure/acrTools';
import { AzureImage } from '../../utils/Azure/models/image';
import { Repository } from '../../utils/Azure/models/repository';
import { NodeBase } from './nodeBase';
import { RegistryType } from './registryType';

Expand Down Expand Up @@ -44,71 +46,20 @@ export class AzureRegistryNode extends NodeBase {

public async getChildren(element: AzureRegistryNode): Promise<AzureRepositoryNode[]> {
const repoNodes: AzureRepositoryNode[] = [];
let node: AzureRepositoryNode;

const tenantId: string = element.subscription.tenantId;
if (!this._azureAccount) {
return [];
}

const session: AzureSession = this._azureAccount.sessions.find((s, i, array) => s.tenantId.toLowerCase() === tenantId.toLowerCase());
const { accessToken, refreshToken } = await acquireToken(session);

if (accessToken && refreshToken) {
let refreshTokenARC;
let accessTokenARC;

await request.post('https://' + element.label + '/oauth2/exchange', {
form: {
grant_type: 'access_token_refresh_token',
service: element.label,
tenant: tenantId,
refresh_token: refreshToken,
access_token: accessToken
}
}, (err, httpResponse, body) => {
if (body.length > 0) {
refreshTokenARC = JSON.parse(body).refresh_token;
} else {
return [];
}
});

await request.post('https://' + element.label + '/oauth2/token', {
form: {
grant_type: 'refresh_token',
service: element.label,
scope: 'registry:catalog:*',
refresh_token: refreshTokenARC
}
}, (err, httpResponse, body) => {
if (body.length > 0) {
accessTokenARC = JSON.parse(body).access_token;
} else {
return [];
}
});
await request.get('https://' + element.label + '/v2/_catalog', {
auth: {
bearer: accessTokenARC
}
}, (err, httpResponse, body) => {
if (body.length > 0) {
const repositories = JSON.parse(body).repositories;
// tslint:disable-next-line:prefer-for-of // Grandfathered in
for (let i = 0; i < repositories.length; i++) {
node = new AzureRepositoryNode(repositories[i], "azureRepositoryNode");
node.accessTokenARC = accessTokenARC;
node.azureAccount = element.azureAccount;
node.refreshTokenARC = refreshTokenARC;
node.registry = element.registry;
node.repository = element.label;
node.subscription = element.subscription;
repoNodes.push(node);
}
}
});
const repositories: Repository[] = await getRepositoriesByRegistry(element.registry);
for (let repo of repositories) {
let node = new AzureRepositoryNode(repo.name, "azureRepositoryNode");
node.azureAccount = element.azureAccount;
node.registry = element.registry;
node.subscription = element.subscription;
repoNodes.push(node);
}

//Note these are ordered by default in alphabetical order
return repoNodes;
}
Expand All @@ -126,11 +77,8 @@ export class AzureRepositoryNode extends NodeBase {
super(label);
}

public accessTokenARC: string;
public azureAccount: AzureAccount
public refreshTokenARC: string;
public registry: ContainerModels.Registry;
public repository: string;
public subscription: SubscriptionModels.Subscription;
public parent: NodeBase;

Expand All @@ -146,94 +94,38 @@ export class AzureRepositoryNode extends NodeBase {
public async getChildren(element: AzureRepositoryNode): Promise<AzureImageNode[]> {
const imageNodes: AzureImageNode[] = [];
let node: AzureImageNode;
let created: string = '';
let refreshTokenARC;
let accessTokenARC;
let tags;

const tenantId: string = element.subscription.tenantId;
const session: AzureSession = element.azureAccount.sessions.find((s, i, array) => s.tenantId.toLowerCase() === tenantId.toLowerCase());
const { accessToken, refreshToken } = await acquireToken(session);

if (accessToken && refreshToken) {
await request.post('https://' + element.repository + '/oauth2/exchange', {
form: {
grant_type: 'access_token_refresh_token',
service: element.repository,
tenant: tenantId,
refresh_token: refreshToken,
access_token: accessToken
}
}, (err, httpResponse, body) => {
if (body.length > 0) {
refreshTokenARC = JSON.parse(body).refresh_token;
} else {
return [];
let repo = new Repository(element.registry, element.label);
let images: AzureImage[] = await getImagesByRepository(repo);
let { acrAccessToken } = await acquireACRAccessTokenFromRegistry(element.registry, `repository:${element.label}:pull`)
const pool = new AsyncPool(MAX_CONCURRENT_REQUESTS);
for (let image of images) {
pool.addTask(async () => {
let data: string;
try {
data = await getRawImageManifest(image, acrAccessToken);
} catch (error) {
vscode.window.showErrorMessage(parseError(error).message);
}
});

await request.post('https://' + element.repository + '/oauth2/token', {
form: {
grant_type: 'refresh_token',
service: element.repository,
scope: 'repository:' + element.label + ':pull',
refresh_token: refreshTokenARC
}
}, (err, httpResponse, body) => {
if (body.length > 0) {
accessTokenARC = JSON.parse(body).access_token;
} else {
return [];
if (data) {
//Acquires each image's manifest to acquire build time.
let manifest = JSON.parse(data);
node = new AzureImageNode(`${element.label}:${image.tag}`, 'azureImageNode');
node.azureAccount = element.azureAccount;
node.registry = element.registry;
node.serverUrl = element.registry.loginServer;
node.subscription = element.subscription;
node.parent = element;
node.created = moment(new Date(JSON.parse(manifest.history[0].v1Compatibility).created)).fromNow();
imageNodes.push(node);
}
});

await request.get('https://' + element.repository + '/v2/' + element.label + '/tags/list', {
auth: {
bearer: accessTokenARC
}
}, (err, httpResponse, body) => {
if (err) { return []; }
if (body.length > 0) {
tags = JSON.parse(body).tags;
}
});

const pool = new AsyncPool(MAX_CONCURRENT_REQUESTS);
// tslint:disable-next-line:prefer-for-of // Grandfathered in
for (let i = 0; i < tags.length; i++) {
pool.addTask(async () => {
let data: string;
try {
data = await request.get('https://' + element.repository + '/v2/' + element.label + `/manifests/${tags[i]}`, {
auth: {
bearer: accessTokenARC
}
});
} catch (error) {
vscode.window.showErrorMessage(parseError(error).message);
}

if (data) {
//Acquires each image's manifest to acquire build time.
let manifest = JSON.parse(data);
node = new AzureImageNode(`${element.label}:${tags[i]}`, 'azureImageNode');
node.azureAccount = element.azureAccount;
node.registry = element.registry;
node.serverUrl = element.repository;
node.subscription = element.subscription;
node.parent = element;
node.created = moment(new Date(JSON.parse(manifest.history[0].v1Compatibility).created)).fromNow();
imageNodes.push(node);
}
});
}
await pool.runAll();

}
function sortFunction(a: AzureImageNode, b: AzureImageNode): number {
await pool.runAll();
function compareFn(a: AzureImageNode, b: AzureImageNode): number {
return a.created.localeCompare(b.created);
}
imageNodes.sort(sortFunction);
imageNodes.sort(compareFn);
return imageNodes;
}
}
Expand Down Expand Up @@ -295,21 +187,3 @@ export class AzureLoadingNode extends NodeBase {
}
}
}

async function acquireToken(session: AzureSession): Promise<{ accessToken: string; refreshToken: string; }> {
return new Promise<{ accessToken: string; refreshToken: string; }>((resolve, reject) => {
const credentials: any = session.credentials;
const environment: any = session.environment;
// tslint:disable-next-line:no-function-expression // Grandfathered in
credentials.context.acquireToken(environment.activeDirectoryResourceId, credentials.username, credentials.clientId, function (err: any, result: { accessToken: string; refreshToken: string; }): void {
if (err) {
reject(err);
} else {
resolve({
accessToken: result.accessToken,
refreshToken: result.refreshToken
});
}
});
});
}
45 changes: 7 additions & 38 deletions explorer/models/registryRootNode.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { parseError } from 'vscode-azureextensionui';
import { keytarConstants, MAX_CONCURRENT_REQUESTS, MAX_CONCURRENT_SUBSCRIPTON_REQUESTS } from '../../constants';
import { AzureAccount } from '../../typings/azure-account.api';
import { AsyncPool } from '../../utils/asyncpool';
import { AzureUtilityManager } from '../../utils/azureUtilityManager';
import * as dockerHub from '../utils/dockerHubUtils'
import { getCoreNodeModule } from '../utils/utils';
import { AzureLoadingNode, AzureNotSignedInNode, AzureRegistryNode } from './azureRegistryNodes';
Expand Down Expand Up @@ -133,19 +134,18 @@ export class RegistryRootNode extends NodeBase {
}

if (loggedIntoAzure) {
const subs: SubscriptionModels.Subscription[] = this.getFilteredSubscriptions();
const subscriptions: SubscriptionModels.Subscription[] = AzureUtilityManager.getInstance().getFilteredSubscriptionList();

const subPool = new AsyncPool(MAX_CONCURRENT_SUBSCRIPTON_REQUESTS);
let subsAndRegistries: { 'subscription': SubscriptionModels.Subscription, 'registries': ContainerModels.RegistryListResult }[] = [];
//Acquire each subscription's data simultaneously
// tslint:disable-next-line:prefer-for-of // Grandfathered in
for (let i = 0; i < subs.length; i++) {
for (let sub of subscriptions) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx. :-)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np :)

subPool.addTask(async () => {
const client = new ContainerRegistryManagement(this.getCredentialByTenantId(subs[i].tenantId), subs[i].subscriptionId);
const client = AzureUtilityManager.getInstance().getContainerRegistryManagementClient(sub);
try {
let regs: ContainerModels.Registry[] = await client.registries.list();
subsAndRegistries.push({
'subscription': subs[i],
'subscription': sub,
'registries': regs
});
} catch (error) {
Expand Down Expand Up @@ -181,42 +181,11 @@ export class RegistryRootNode extends NodeBase {
}
await regPool.runAll();

function sortFunction(a: AzureRegistryNode, b: AzureRegistryNode): number {
function compareFn(a: AzureRegistryNode, b: AzureRegistryNode): number {
return a.registry.loginServer.localeCompare(b.registry.loginServer);
}
azureRegistryNodes.sort(sortFunction);
azureRegistryNodes.sort(compareFn);
return azureRegistryNodes;
}
}

private getCredentialByTenantId(tenantId: string): ServiceClientCredentials {

const session = this._azureAccount.sessions.find((s, i, array) => s.tenantId.toLowerCase() === tenantId.toLowerCase());

if (session) {
return session.credentials;
}

throw new Error(`Failed to get credentials, tenant ${tenantId} not found.`);
}

private getFilteredSubscriptions(): SubscriptionModels.Subscription[] {

if (this._azureAccount) {
return this._azureAccount.filters.map<SubscriptionModels.Subscription>(filter => {
return {
id: filter.subscription.id,
session: filter.session,
subscriptionId: filter.subscription.subscriptionId,
tenantId: filter.session.tenantId,
displayName: filter.subscription.displayName,
state: filter.subscription.state,
subscriptionPolicies: filter.subscription.subscriptionPolicies,
authorizationSource: filter.subscription.authorizationSource
};
});
} else {
return [];
}
}
}
19 changes: 16 additions & 3 deletions utils/Azure/acrTools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ import { AzureImage } from "./models/image";
import { Repository } from "./models/repository";

//General helpers
/**
* @param registry gets the subscription for a given registry
/** Gets the subscription for a given registry
* @returns a subscription object
*/
export function getSubscriptionFromRegistry(registry: Registry): SubscriptionModels.Subscription {
Expand All @@ -26,7 +25,7 @@ export function getSubscriptionFromRegistry(registry: Registry): SubscriptionMod
return subscription;
}

export function getResourceGroupName(registry: Registry): any {
export function getResourceGroupName(registry: Registry): string {
return registry.id.slice(registry.id.search('resourceGroups/') + 'resourceGroups/'.length, registry.id.search('/providers/'));
}

Expand Down Expand Up @@ -55,6 +54,20 @@ export async function getImagesByRepository(element: Repository): Promise<AzureI
return allImages;
}

/** Returns the textual manifest for an ACR Image
* @param acrAccessToken If the token is not provided one will be created for the particular image, if doing multiple requests this should be provided
*/
export async function getRawImageManifest(image: AzureImage, acrAccessToken?: string): Promise<string> {
if (!acrAccessToken) {
acrAccessToken = (await acquireACRAccessTokenFromRegistry(image.registry, `repository:${image.repository}:pull`)).acrAccessToken;
}
return await request.get('https://' + image.registry.loginServer + '/v2/' + image.repository.name + `/manifests/${image.tag}`, {
auth: {
bearer: acrAccessToken
}
});
}

/** List repositories on a given Registry. */
export async function getRepositoriesByRegistry(registry: Registry): Promise<Repository[]> {
const allRepos: Repository[] = [];
Expand Down