Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add-geolocation-information-to-location #105

Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,27 +14,23 @@ On the one hand, the internet offers many ways to find out about Berlin’s dive

kulturdaten.berlin is a project of the Technologiestiftung Berlin (<https://www.technologiestiftung-berlin.de>), funded by the Senate Department for Culture and Social Cohesion.


## Links

- [ReadMe API Docs](https://kulturdaten.readme.io)
- [Swagger API Docs](https://api-v2.kulturdaten.berlin/api/docs/)
- [OpenAPI YML file](https://github.com/technologiestiftung/kulturdaten-api/blob/main/src/schemas/kulturdaten.berlin.openapi.generated.yml)


## Prerequisites

- Install a Node version according to the version specified in `.nvmrc` (e.g. with `nvm install` or `nvm use`)
- Install [MongoDB Community Edition](https://www.mongodb.com/docs/manual/administration/install-community/)


## Installation

1. Install packages with `npm install`
2. Generate OpenAPI schema files in `/src/generated` with `npm run generate`
3. Create an `.env` in the root directory and fill it with values (see `.env.example` for inspiration)


## Development

```shell
Expand All @@ -45,7 +41,6 @@ npm run dev
npm run debug
```


## More commands

```shell
Expand All @@ -65,7 +60,6 @@ npm run lint-openapi
npm run typecheck
```


## Tests

Run all tests with Vitest (incl. hot reload when run locally):
Expand All @@ -74,14 +68,12 @@ Run all tests with Vitest (incl. hot reload when run locally):
npm test
```


## Production Build

```shell
npm start
```


## Initializing an empty database

The Seeder.ts script assists in initializing your database. Use the -t or --tags flag to add predefined tags. For setting up organizations for Berlin boroughs (excluding 'außerhalb'), use the -b or --boroughs flag with a default password. To set up an admin user, utilize the -a or --admin flag followed by credentials in email:password format. Combine flags for simultaneous operations. Example:
Expand All @@ -92,12 +84,10 @@ npm run seed -- --tags --boroughs password123 --admin admin@example.com:password

Note: The script only populates empty collections to avoid duplicates and unintended overwrites, especially important for admin user creation.


## Contributing

Before you create a pull request, write an issue so we can discuss your changes.


## Contributors

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
Expand All @@ -123,12 +113,10 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!


## Content Licensing

Text and content available as [CC BY](https://creativecommons.org/licenses/by/3.0/de/).


## Credits

<table>
Expand Down
30 changes: 21 additions & 9 deletions src/admin/AdminRoutes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,40 @@ import { AttractionsController } from "../resources/attractions/controllers/Attr
import { Permit } from "../resources/auth/middleware/Permit";
import { getPagination } from "../utils/RequestUtil";
import { DistrictDataHarvestersController } from "./districtData/controllers/DistrictDataHarvestersController";
import { CoordinatesToLocationsController } from "./dataEnrichment/controllers/CoordinatesToLocationsController";

const log: debug.IDebugger = debug("app:admin-routes");

@Service()
export class AdminRoutes {
constructor(
public districtDataHarvestersController: DistrictDataHarvestersController,
public coordinatesToLocationsController: CoordinatesToLocationsController,
public attractionsController: AttractionsController,
) {}

public getRouter(): Router {
const router = express.Router();

router.post(
"/harvest/baevents-bezirkskalender",
passport.authenticate("authenticated-user", { session: false }),
Permit.authorizesAsAdmin(),
(req: express.Request, res: express.Response) => {
const calendarIDs = req.body as string[];
this.districtDataHarvestersController.harvest(res, calendarIDs);
},
);
router
.post(
"/harvest/baevents-bezirkskalender",
passport.authenticate("authenticated-user", { session: false }),
Permit.authorizesAsAdmin(),
(req: express.Request, res: express.Response) => {
const calendarIDs = req.body as string[];
this.districtDataHarvestersController.harvest(res, calendarIDs);
},
)
.post(
"/data-enrichment/coordinates-to-locations",
passport.authenticate("authenticated-user", { session: false }),
Permit.authorizesAsAdmin(),
(req: express.Request, res: express.Response) => {
const locationsIDs = req.body as string[];
this.coordinatesToLocationsController.enrichData(res, locationsIDs);
},
);

router
.get(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import { Response } from "express";
import { Service } from "typedi";
import { SuccessResponseBuilder } from "../../../common/responses/SuccessResponseBuilder";
import { CoordinatesToLocationsResponse } from "../../../generated/models/CoordinatesToLocationsResponse.generated";
import { CoordinatesToLocationsService } from "../services/CoordinatesToLocationsService";

@Service()
export class CoordinatesToLocationsController {
constructor(public service: CoordinatesToLocationsService) {}

async enrichData(res: Response, locationsIDs: string[]) {

Check warning on line 12 in src/admin/dataEnrichment/controllers/CoordinatesToLocationsController.ts

View workflow job for this annotation

GitHub Actions / linter-and-typescript

Delete `⏎`
const enrichedLocations = await this.service.enrichData(locationsIDs);

res.status(200).send(
new SuccessResponseBuilder<CoordinatesToLocationsResponse>()
.okResponse({
enrichedLocations: enrichedLocations,
})
.build(),
);
}
}
27 changes: 27 additions & 0 deletions src/admin/dataEnrichment/services/CoordinatesToLocationsService.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { Inject, Service } from "typedi";
import { Location } from "../../../generated/models/Location.generated";
import { LocationsRepository } from "../../../resources/locations/repositories/LocationsRepository";
import { MongoDBFilterFactory } from "../../../common/filter/FilterFactory";

@Service()
export class CoordinatesToLocationsService {
constructor(@Inject("LocationsRepository") public locationsRepository: LocationsRepository) {}

async enrichData(locationsIDs: string[]): Promise<string[]> {
const filterFactory = new MongoDBFilterFactory();
let locationsToBeEnriched: Location[] = [];
if (locationsIDs.length > 0) {
locationsToBeEnriched = await this.locationsRepository.getLocations(
undefined,
filterFactory.createAnyMatchFilter("identifier", locationsIDs),
);
} else {
locationsToBeEnriched = await this.locationsRepository.getLocations(
undefined,
filterFactory.createNotExistsFilter("coordinates"),
);
}

return locationsToBeEnriched.map((location) => location.identifier);
}
}
3 changes: 3 additions & 0 deletions src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import { TagsRoutes } from "./resources/tags/tags.routes";
import { UsersRoutes } from "./resources/users/UsersRoutes";
import { MongoDBUsersRepository } from "./resources/users/repositories/MongoDBUsersRepository";
import { UsersService } from "./resources/users/services/UsersService";
import { MongoDBFilterFactory } from "./common/filter/FilterFactory";

export class KulturdatenBerlinApp {
constructor(public app: express.Application) {}
Expand Down Expand Up @@ -115,6 +116,8 @@ export class KulturdatenBerlinApp {

Container.set("TagsRepository", new MongoDBTagsRepository(Container.get("Database")));
Container.import([]);

Container.set("FilterFactory", new MongoDBFilterFactory());
}

private initLogger() {
Expand Down
111 changes: 111 additions & 0 deletions src/common/filter/FilterFactory.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
import { Service } from "typedi";
import { Filter } from "../../generated/models/Filter.generated";

export interface FilterFactory {
createExactMatchFilter(propertyName: string, value: string | undefined): Filter | undefined;

createPartialMatchFilter(propertyName: string, value: string | undefined): Filter | undefined;

createNotEqualFilter(propertyName: string, value: string | undefined): Filter | undefined;

createAnyMatchFilter(propertyName: string, values: string[] | undefined): Filter | undefined;

createAllMatchFilter(propertyName: string, values: string[] | undefined): Filter | undefined;

createFutureDateFilter(propertyName: string): Filter | undefined;

createDateRangeFilter(
startDateProperty: string,
endDateProperty: string,
startDate?: string,
endDate?: string,
): Filter | undefined;

combineWithAnd(filters: (Filter | undefined)[]): Filter;

combineWithOr(filters: (Filter | undefined)[]): Filter;
}

@Service()
export class MongoDBFilterFactory implements FilterFactory {
createExactMatchFilter(propertyName: string, value: string | undefined): Filter | undefined {
if (!value) {
return undefined;
}
return { [propertyName as string]: value };
}

createPartialMatchFilter(propertyName: string, value: string | undefined): Filter | undefined {
if (!value) {
return undefined;
}
return { [propertyName as string]: { $regex: value, $options: "i" } };
}

createNotEqualFilter(propertyName: string, value: string | undefined): Filter | undefined {
if (value === undefined) {
return undefined;
}
return { [propertyName]: { $ne: value } };
}

createAnyMatchFilter(propertyName: string, values: string[] | undefined): Filter | undefined {
if (!values) {
return undefined;
}
return { [propertyName as string]: { $in: values } };
}

createAllMatchFilter(propertyName: string, values: string[] | undefined): Filter | undefined {
if (!values) {
return undefined;
}
return { [propertyName as string]: { $all: values } };
}

createFutureDateFilter(propertyName: string): Filter | undefined {
const currentDate = new Date().toISOString().split("T")[0]; // YYYY-MM-DD Format
return { [propertyName]: { $gte: currentDate } };
}

createNotExistsFilter(propertyName: string): Filter | undefined {
return { [propertyName]: { $exists: false } };
}

createDateRangeFilter(
startDateProperty: string,
endDateProperty: string,
startDate?: string,
endDate?: string,
): Filter | undefined {
if (startDate && endDate) {
return {
$and: [{ [startDateProperty]: { $gte: startDate } }, { [endDateProperty]: { $lte: endDate } }],
};
} else if (startDate) {
return { [startDateProperty]: { $gte: startDate } };
} else if (endDate) {
return { [endDateProperty]: { $lte: endDate } };
} else {
return undefined;
}
}

combineWithAnd(filters: (Filter | undefined)[]): Filter {
const validFilters = filters.filter((f) => f && Object.keys(f).length > 0) as Filter[];
if (validFilters.length > 0) {
return { $and: validFilters };
} else {
return {};
}
}

combineWithOr(filters: (Filter | undefined)[]): Filter {
const validFilters = filters.filter((f) => f && Object.keys(f).length > 0) as Filter[];
if (validFilters.length > 0) {
return { $or: validFilters };
} else {
return {};
}
}
}
43 changes: 43 additions & 0 deletions src/common/filter/FilterFactory.unit.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import { FilterFactory, MongoDBFilterFactory } from "./FilterFactory";

describe("MongoDBFilterFactory", () => {
let factory: FilterFactory;

beforeEach(() => {
factory = new MongoDBFilterFactory();
});

it("should create an exact match filter", () => {
const filter = factory.createExactMatchFilter("identifier", "123");
expect(filter).toEqual({ identifier: "123" });
});

it("should create a partial match filter", () => {
const filter = factory.createPartialMatchFilter("identifier", "123");
expect(filter).toEqual({ identifier: { $regex: "123", $options: "i" } });
});

it("should create an any match filter", () => {
const filter = factory.createAnyMatchFilter("tags", ["tag1", "tag2"]);
expect(filter).toEqual({ tags: { $in: ["tag1", "tag2"] } });
});

it("should create an all match filter", () => {
const filter = factory.createAllMatchFilter("tags", ["tag1", "tag2"]);
expect(filter).toEqual({ tags: { $all: ["tag1", "tag2"] } });
});

it("should combine filters with AND", () => {
const filter1 = { identifier: "123" };
const filter2 = { type: "type.Attraction" };
const combinedFilter = factory.combineWithAnd([filter1, filter2]);
expect(combinedFilter).toEqual({ $and: [filter1, filter2] });
});

it("should combine filters with OR", () => {
const filter1 = { identifier: "123" };
const filter2 = { type: "type.Attraction" };
const combinedFilter = factory.combineWithOr([filter1, filter2]);
expect(combinedFilter).toEqual({ $or: [filter1, filter2] });
});
});
34 changes: 33 additions & 1 deletion src/common/parameters/Params.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,35 @@
export type Params = {
[key: string]: string;
[key: string]: string | string[] | boolean | undefined;
};

export type AttractionParams = Params & {
curatedBy?: string;
editableBy?: string;
anyTags?: string[];
allTags?: string[];
};

export type EventParams = Params & {
asReference?: string;
organizedBy?: string;
editableBy?: string;
byLocation?: string;
byAttraction?: string;
isFreeOfCharge?: boolean;
inFuture?: boolean;
startDate?: string;
endDate?: string;
};

export type LocationParams = Params & {
asReference?: string;
managedBy?: string;
editableBy?: string;
anyAccessibilities?: string[];
allAccessibilities?: string[];
};

export type OrganizationParams = Params & {
asReference?: string;
editableBy?: string;
};
Loading
Loading