Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linguist - Removed Deprecated Code #1804

Merged
merged 4 commits into from
Nov 15, 2024
Merged

Linguist - Removed Deprecated Code #1804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d83adc8
Linguist - Removed Deprecated Code
awanlin Oct 31, 2024
6b2bfca
Fixed typo
awanlin Oct 31, 2024
02368da
Removed export
awanlin Nov 1, 2024
facb3f8
Merge branch 'main' into topic/linguist-remove-deprecated-code
awanlin Nov 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions workspaces/linguist/.changeset/dry-teachers-tap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@backstage-community/plugin-linguist-backend': minor
---

**BREAKING** Removed support for what is known as the legacy backend, please use the New Backend System.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol @ "minor breaking" changes which are not considered major :P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not yet 1.0, I'm following this: https://github.com/backstage/backstage/blob/master/REVIEWING.md#reviewing-changeset-bump-levels

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we ship as a major? It could make sense

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to keep following the conventions I've been following so far.

2 changes: 0 additions & 2 deletions workspaces/linguist/packages/backend/knip-report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@
| @backstage/plugin-permission-common | package.json | error |
| @backstage/plugin-permission-node | package.json | error |
| @backstage/plugin-auth-node | package.json | error |
| @backstage/backend-common | package.json | error |
| @backstage/backend-tasks | package.json | error |
| @backstage/config | package.json | error |
| better-sqlite3 | package.json | error |
| dockerode | package.json | error |
Expand Down
1 change: 0 additions & 1 deletion workspaces/linguist/packages/backend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
"@backstage-community/plugin-catalog-backend-module-linguist-tags-processor": "workspace:^",
"@backstage-community/plugin-linguist-backend": "workspace:^",
"@backstage/backend-defaults": "^0.5.2",
"@backstage/backend-tasks": "^0.6.1",
"@backstage/config": "^1.2.0",
"@backstage/plugin-app-backend": "^0.3.76",
"@backstage/plugin-auth-backend": "^0.23.1",
Expand Down
7 changes: 0 additions & 7 deletions .../linguist/plugins/catalog-backend-module-linguist-tags-processor/knip-report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,3 @@
| Name | Location | Severity |
| :------------------------------------------ | :----------- | :------- |
| @backstage-community/plugin-linguist-common | package.json | error |
| @backstage/backend-common | package.json | error |

## Unused devDependencies (1)

| Name | Location | Severity |
| :----------------------- | :----------- | :------- |
| @backstage/backend-tasks | package.json | error |
1 change: 0 additions & 1 deletion workspaces/linguist/plugins/catalog-backend-module-linguist-tags-processor/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
"node-fetch": "^2.6.7"
},
"devDependencies": {
"@backstage/backend-tasks": "^0.6.1",
"@backstage/backend-test-utils": "^1.0.2",
"@backstage/cli": "^0.28.0",
"js-yaml": "^4.1.0",
Expand Down
155 changes: 64 additions & 91 deletions workspaces/linguist/plugins/linguist-backend/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Linguist Backend

Welcome to the Linguist backend plugin! This plugin provides data for the Linguist frontend features. Additionally, it provides an optional entity processor which will automate adding language tags to your entities.
Welcome to the Linguist backend plugin! This plugin provides data for the Linguist frontend features.

## Setup

Expand All @@ -17,136 +17,109 @@ Here's how to get the backend up and running:
yarn --cwd packages/backend add @backstage-community/plugin-linguist-backend
```

2. Then we will create a new file named `packages/backend/src/plugins/linguist.ts`, and add the
following to it:

```ts
import { TaskScheduleDefinition } from '@backstage/backend-tasks';
import { createRouter } from '@backstage-community/plugin-linguist-backend';
import { Router } from 'express';
import type { PluginEnvironment } from '../types';

export default async function createPlugin(
env: PluginEnvironment,
): Promise<Router> {
const schedule: TaskScheduleDefinition = {
frequency: { minutes: 2 },
timeout: { minutes: 15 },
initialDelay: { seconds: 90 },
};

return createRouter({ schedule: schedule }, { ...env });
}
```
2. Then in your `packages/backend/src/index.ts` make the following changes:

3. Next we wire this into the overall backend router, edit `packages/backend/src/index.ts`:

```ts
import linguist from './plugins/linguist';
// ...
async function main() {
// ...
// Add this line under the other lines that follow the useHotMemoize pattern
const linguistEnv = useHotMemoize(module, () => createEnv('linguist'));
// ...
// Insert this line under the other lines that add their routers to apiRouter in the same way
apiRouter.use('/linguist', await linguist(linguistEnv));
```
```diff
import { createBackend } from '@backstage/backend-defaults';

4. Now run `yarn start-backend` from the repo root
5. Finally open `http://localhost:7007/api/linguist/health` in a browser and it should return `{"status":"ok"}`
const backend = createBackend();

#### New Backend System
// ... other feature additions

The Linguist backend plugin has support for the [new backend system](https://backstage.io/docs/backend-system/), here's how you can set that up:
+ backend.add(import('@backstage-community/plugin-linguist-backend'));

In your `packages/backend/src/index.ts` make the following changes:
backend.start();
```

```diff
import { createBackend } from '@backstage/backend-defaults';
3. Then update your `app-config.yaml` with your desired options, more details in the [Plugin Options](#plugin-options) section below, here's a basic example to get started:

```yaml
# ... other config

linguist:
schedule:
frequency:
minutes: 2
timeout:
minutes: 2
initialDelay:
seconds: 15
age:
days: 30
batchSize: 2
useSourceLocation: false
# ... other config
```

const backend = createBackend();
4. Now run `yarn start-backend` from the repo root
5. Finally open `http://localhost:7007/api/linguist/health` in a browser and it should return `{"status":"ok"}`

// ... other feature additions
## Plugin Options

+ backend.add(import('@backstage-community/plugin-linguist-backend'));
The Linguist backend has various plugin options that you can provide in the `app-config.yaml` file that will allow you to configure various aspects of how it works. The following sections go into the details of these options

backend.start();
```
### Batch Size

The plugin options can be set through the `app-config.yaml`:
The Linguist backend is setup to process entities by acting as a queue where it will pull down all the applicable entities from the Catalog and add them to it's database (saving just the `entityRef`). Then it will grab the `n` oldest entities that have not been processed to determine their languages and process them. To control the batch size simply provide that to the `app-config.yaml` file like this:

```yaml
// ...
# ... other config

linguist:
schedule:
frequency:
minutes: 2
timeout:
minutes: 2
initialDelay:
seconds: 15
age:
days: 30
batchSize: 2
useSourceLocation: false

// ...
```

## Plugin Option

The Linguist backend has various plugin options that you can provide to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` file that will allow you to configure various aspects of how it works. The following sections go into the details of these options

### Batch Size

The Linguist backend is setup to process entities by acting as a queue where it will pull down all the applicable entities from the Catalog and add them to it's database (saving just the `entityRef`). Then it will grab the `n` oldest entities that have not been processed to determine their languages and process them. To control the batch size simply provide that to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` like this:

```ts
return createRouter({ schedule: schedule, batchSize: 40 }, { ...env });
# ... other config
```

**Note:** The default batch size is 20

### Kind

The default setup only processes entities of kind `['API', 'Component', 'Template']`. To control the `kind` that are processed provide that to the `createRouter` function in your `packages/backend/src/plugins/linguist.ts` like this:
The default setup only processes entities of kind `['API', 'Component', 'Template']`. To control the `kind` that are processed add that in your `app-config.yaml` file like this:

```yaml
# ... other config

```ts
return createRouter({ schedule: schedule, kind: ['Component'] }, { ...env });
linguist:
kind: ['Component']
# ... other config
```

### Refresh

The default setup will only generate the language breakdown for entities with the linguist annotation that have not been generated yet. If you want this process to also refresh the data you can do so by adding the `age` (as a `HumanDuration`) in your `packages/backend/src/plugins/linguist.ts` when you call `createRouter`:
The default setup will only generate the language breakdown for entities with the linguist annotation that have not been generated yet. If you want this process to also refresh the data you can do so by adding the `age` (as a `HumanDuration`) in your `app-config.yaml` file like this:

```yaml
# ... other config

```ts
return createRouter({ schedule: schedule, age: { days: 30 } }, { ...env });
linguist:
age: { days: 30 }
# ... other config
```

With the `age` setup like this if the language breakdown is older than 15 days it will get regenerated. It's recommended that if you choose to use this configuration to set it to a large value - 30, 90, or 180 - as this data generally does not change drastically.

### Linguist JS options

The default setup will use the default [linguist-js](https://www.npmjs.com/package/linguist-js) options, a full list of the available options can be found [here](https://www.npmjs.com/package/linguist-js#API).
The default setup will use the default [linguist-js](https://www.npmjs.com/package/linguist-js) options, a full list of the available options can be found [here](https://www.npmjs.com/package/linguist-js#API). Here's a simple example of using this in your `app-config.yaml` file:

```yaml
# ... other config

```ts
return createRouter(
{ schedule: schedule, linguistJsOptions: { offline: true } },
{ ...env },
);
linguist:
linguistJsOptions: { offline: true }
# ... other config
```

### Use Source Location

You may wish to use the `backstage.io/source-location` annotation over using the `backstage.io/linguist` as you may not be able to quickly add that annotation to your Entities. To do this you'll just need to set the `useSourceLocation` boolean to `true` in your `packages/backend/src/plugins/linguist.ts` when you call `createRouter`:
You may wish to use the `backstage.io/source-location` annotation over using the `backstage.io/linguist` as you may not be able to quickly add that annotation to your Entities. To do this you'll just need to set the `useSourceLocation` boolean to `true` in your `app-config.yaml` file:

```ts
return createRouter(
{ schedule: schedule, useSourceLocation: true },
{ ...env },
);
```yaml
# ... other config

linguist:
useSourceLocation: true
# ... other config
```

**Note:** This has the potential to cause a lot of processing, be very thoughtful about this before hand
Expand Down
34 changes: 2 additions & 32 deletions workspaces/linguist/plugins/linguist-backend/config.d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@
* limitations under the License.
*/

import { TaskScheduleDefinition } from '@backstage/backend-tasks';
import { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api';
import { HumanDuration } from '@backstage/types';
import { Options as LinguistJsOptions } from 'linguist-js/dist/types';

export interface Config {
/** Configuration options for the linguist plugin */
linguist?: {
schedule?: TaskScheduleDefinition;
schedule?: SchedulerServiceTaskScheduleDefinition;
/**
* @default 20
*/
Expand All @@ -42,35 +42,5 @@ export interface Config {
* [linguist-js](https://www.npmjs.com/package/linguist-js) options
*/
linguistJsOptions?: LinguistJsOptions;

/** Options for the tags processor */
tagsProcessor?: {
/**
* Determines how many bytes of a language should be in a repo
* for it to be added as an entity tag. Defaults to 0.
*/
bytesThreshold?: number;
/**
* The types of linguist languages that should be processed. Can be
* any of "programming", "data", "markup", "prose". Defaults to ["programming"].
*/
languageTypes?: string[];
/**
* A custom mapping of linguist languages to how they should be rendered as entity tags.
* If a language is mapped to '' it will not be included as a tag.
*/
languageMap?: {
[language: string]: string | undefined;
};
/**
* How long to cache entity languages for in memory. Used to avoid constant db hits during
* processing. Defaults to 30 minutes.
*/
cacheTTL?: HumanDuration;
/**
* An optional prefix to apply to all created tags from linguist
*/
tagPrefix?: string;
};
};
}
1 change: 0 additions & 1 deletion workspaces/linguist/plugins/linguist-backend/knip-report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,4 @@
| Name | Location | Severity |
| :------------------------------------------ | :----------- | :------- |
| @backstage-community/plugin-linguist-common | package.json | error |
| @backstage/backend-tasks | package.json | error |
| yn | package.json | error |
2 changes: 0 additions & 2 deletions workspaces/linguist/plugins/linguist-backend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,8 @@
},
"dependencies": {
"@backstage-community/plugin-linguist-common": "workspace:^",
"@backstage/backend-common": "^0.25.0",
"@backstage/backend-defaults": "^0.5.2",
"@backstage/backend-plugin-api": "^1.0.1",
"@backstage/backend-tasks": "^0.6.1",
"@backstage/catalog-client": "^1.7.1",
"@backstage/catalog-model": "^1.7.0",
"@backstage/config": "^1.2.0",
Expand Down
60 changes: 0 additions & 60 deletions workspaces/linguist/plugins/linguist-backend/report.api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,30 +3,8 @@
> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).

```ts
import { AuthService } from '@backstage/backend-plugin-api';
import { BackendFeature } from '@backstage/backend-plugin-api';
import { Config } from '@backstage/config';
import { DatabaseService } from '@backstage/backend-plugin-api';
import { DiscoveryService } from '@backstage/backend-plugin-api';
import express from 'express';
import { HttpAuthService } from '@backstage/backend-plugin-api';
import { HumanDuration } from '@backstage/types';
import { Languages } from '@backstage-community/plugin-linguist-common';
import { LoggerService } from '@backstage/backend-plugin-api';
import { SchedulerService } from '@backstage/backend-plugin-api';
import { SchedulerServiceTaskScheduleDefinition } from '@backstage/backend-plugin-api';
import { UrlReaderService } from '@backstage/backend-plugin-api';

// @public @deprecated (undocumented)
export function createRouter(
pluginOptions: PluginOptions,
routerOptions: RouterOptions,
): Promise<express.Router>;

// @public @deprecated (undocumented)
export function createRouterFromConfig(
routerOptions: RouterOptions,
): Promise<express.Router>;

// @public (undocumented)
export interface LinguistBackendApi {
Expand All @@ -39,42 +17,4 @@ export interface LinguistBackendApi {
// @public
const linguistPlugin: BackendFeature;
export default linguistPlugin;

// @public @deprecated (undocumented)
export interface PluginOptions {
// (undocumented)
age?: HumanDuration;
// (undocumented)
batchSize?: number;
// (undocumented)
kind?: string[];
// (undocumented)
linguistJsOptions?: Record<string, unknown>;
// (undocumented)
schedule?: SchedulerServiceTaskScheduleDefinition;
// (undocumented)
useSourceLocation?: boolean;
}

// @public @deprecated (undocumented)
export interface RouterOptions {
// (undocumented)
auth?: AuthService;
// (undocumented)
config: Config;
// (undocumented)
database: DatabaseService;
// (undocumented)
discovery: DiscoveryService;
// (undocumented)
httpAuth?: HttpAuthService;
// (undocumented)
linguistBackendApi?: LinguistBackendApi;
// (undocumented)
logger: LoggerService;
// (undocumented)
reader: UrlReaderService;
// (undocumented)
scheduler?: SchedulerService;
}
```
Loading
Loading