Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image api rework #5260

Merged
merged 42 commits into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
56ab67f
Split image endpoints into API v3 and v4
Nutomic Dec 12, 2024
05843fb
Move into subfolders
Nutomic Dec 12, 2024
cfa866a
Upload avatar endpoint and other changes
Nutomic Dec 13, 2024
d252be2
Various other changes
Nutomic Dec 13, 2024
e815778
clippy
Nutomic Dec 13, 2024
60ba7af
config options
Nutomic Dec 13, 2024
7c771d2
fix ts bindings
Nutomic Dec 13, 2024
8ae4b40
fix api tests
Nutomic Dec 17, 2024
b0d4bdb
Add option to disable image upload (fixes #1118)
Nutomic Dec 17, 2024
55b8ace
split files into upload, download
Nutomic Dec 17, 2024
460ccc8
move sitemap to top level, not in api
Nutomic Dec 18, 2024
5f06195
simplify code
Nutomic Dec 18, 2024
5f49b2a
add upload user banner
Nutomic Dec 18, 2024
a6f7e76
community icon/banner
Nutomic Dec 18, 2024
ada8f1b
site icon/banner
Nutomic Dec 18, 2024
4951aa0
update js client
Nutomic Dec 18, 2024
2ec8cd3
Merge branch 'main' into image-api-rework
Nutomic Jan 3, 2025
961c7f1
wip
Nutomic Jan 8, 2025
705703f
add delete endpoints
Nutomic Jan 9, 2025
216fca2
change comment
Nutomic Jan 9, 2025
6f91754
Merge branch 'main' into image-api-rework
Nutomic Jan 9, 2025
4d51f53
optimization
Nutomic Jan 9, 2025
96cf017
move fn
Nutomic Jan 9, 2025
f04da5e
1024px banner
Nutomic Jan 9, 2025
7cbbb9a
dont use static client
Nutomic Jan 10, 2025
83c3304
fix api tests
Nutomic Jan 10, 2025
f040f91
shear
Nutomic Jan 10, 2025
6123659
proxy pictrs in request.rs (fixes #5270)
Nutomic Jan 10, 2025
ec74669
clippy
Nutomic Jan 10, 2025
ee091c7
try to fix api tests
Nutomic Jan 10, 2025
76f4508
skip api tests
Nutomic Jan 10, 2025
c9dfbfe
create user
Nutomic Jan 10, 2025
7f78275
debug
Nutomic Jan 10, 2025
2e7a961
dbg
Nutomic Jan 10, 2025
1c12993
test
Nutomic Jan 13, 2025
e484489
image
Nutomic Jan 13, 2025
a45fcc5
run another
Nutomic Jan 13, 2025
4b3f2f5
fixed?
Nutomic Jan 13, 2025
4b94043
Merge branch 'main' into image-api-rework
Nutomic Jan 13, 2025
79cdb83
clippy
Nutomic Jan 13, 2025
2be56b7
fix
Nutomic Jan 13, 2025
47638cc
fix health check
Nutomic Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion api_tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"eslint": "^9.14.0",
"eslint-plugin-prettier": "^5.1.3",
"jest": "^29.5.0",
"lemmy-js-client": "0.20.0-api-v4.16",
"lemmy-js-client": "0.20.0-image-api-rework.6",
"prettier": "^3.2.5",
"ts-jest": "^29.1.0",
"typescript": "^5.5.4",
Expand Down
10 changes: 5 additions & 5 deletions api_tests/pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions api_tests/src/community.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ import {
followCommunity,
banPersonFromCommunity,
resolvePerson,
getSite,
createPost,
getPost,
resolvePost,
Expand All @@ -36,7 +35,7 @@ import {
userBlockInstance,
} from "./shared";
import { AdminAllowInstanceParams } from "lemmy-js-client/dist/types/AdminAllowInstanceParams";
import { EditCommunity, EditSite, GetPosts } from "lemmy-js-client";
import { EditCommunity, GetPosts } from "lemmy-js-client";

beforeAll(setupLogins);
afterAll(unfollows);
Expand Down
1 change: 0 additions & 1 deletion api_tests/src/follow.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import {
setupLogins,
resolveBetaCommunity,
followCommunity,
getSite,
waitUntil,
beta,
betaUrl,
Expand Down
85 changes: 38 additions & 47 deletions api_tests/src/image.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ jest.setTimeout(120000);

import {
UploadImage,
DeleteImage,
PurgePerson,
PurgePost,
DeleteImageParams,
} from "lemmy-js-client";
import {
alpha,
Expand All @@ -18,7 +18,6 @@ import {
epsilon,
followCommunity,
gamma,
getSite,
imageFetchLimit,
registerUser,
resolveBetaCommunity,
Expand Down Expand Up @@ -54,13 +53,12 @@ test("Upload image and delete it", async () => {
image: Buffer.from("test"),
};
const upload = await alphaImage.uploadImage(upload_form);
expect(upload.files![0].file).toBeDefined();
expect(upload.files![0].delete_token).toBeDefined();
expect(upload.url).toBeDefined();
expect(upload.delete_url).toBeDefined();
expect(upload.image_url).toBeDefined();
expect(upload.filename).toBeDefined();
expect(upload.delete_token).toBeDefined();

// ensure that image download is working. theres probably a better way to do this
const response = await fetch(upload.url ?? "");
const response = await fetch(upload.image_url ?? "");
const content = await response.text();
expect(content.length).toBeGreaterThan(0);

Expand All @@ -77,26 +75,21 @@ test("Upload image and delete it", async () => {
const previousThumbnails = 1;
expect(listAllMediaRes.images.length).toBe(previousThumbnails);

// The deleteUrl is a combination of the endpoint, delete token, and alias
let firstImage = listMediaRes.images[0];
let deleteUrl = `${alphaUrl}/pictrs/image/delete/${firstImage.local_image.pictrs_delete_token}/${firstImage.local_image.pictrs_alias}`;
expect(deleteUrl).toBe(upload.delete_url);

// Make sure the uploader is correct
expect(firstImage.person.actor_id).toBe(
expect(listMediaRes.images[0].person.actor_id).toBe(
`http://lemmy-alpha:8541/u/lemmy_alpha`,
);

// delete image
const delete_form: DeleteImage = {
token: upload.files![0].delete_token,
filename: upload.files![0].file,
const delete_form: DeleteImageParams = {
token: upload.delete_token,
filename: upload.filename,
};
const delete_ = await alphaImage.deleteImage(delete_form);
expect(delete_).toBe(true);
expect(delete_.success).toBe(true);

// ensure that image is deleted
const response2 = await fetch(upload.url ?? "");
const response2 = await fetch(upload.image_url ?? "");
const content2 = await response2.text();
expect(content2).toBe("");

Expand All @@ -119,13 +112,12 @@ test("Purge user, uploaded image removed", async () => {
image: Buffer.from("test"),
};
const upload = await user.uploadImage(upload_form);
expect(upload.files![0].file).toBeDefined();
expect(upload.files![0].delete_token).toBeDefined();
expect(upload.url).toBeDefined();
expect(upload.delete_url).toBeDefined();
expect(upload.filename).toBeDefined();
expect(upload.delete_token).toBeDefined();
expect(upload.image_url).toBeDefined();

// ensure that image download is working. theres probably a better way to do this
const response = await fetch(upload.url ?? "");
const response = await fetch(upload.image_url ?? "");
const content = await response.text();
expect(content.length).toBeGreaterThan(0);

Expand All @@ -138,7 +130,7 @@ test("Purge user, uploaded image removed", async () => {
expect(delete_.success).toBe(true);

// ensure that image is deleted
const response2 = await fetch(upload.url ?? "");
const response2 = await fetch(upload.image_url ?? "");
const content2 = await response2.text();
expect(content2).toBe("");
});
Expand All @@ -151,23 +143,22 @@ test("Purge post, linked image removed", async () => {
image: Buffer.from("test"),
};
const upload = await user.uploadImage(upload_form);
expect(upload.files![0].file).toBeDefined();
expect(upload.files![0].delete_token).toBeDefined();
expect(upload.url).toBeDefined();
expect(upload.delete_url).toBeDefined();
expect(upload.filename).toBeDefined();
expect(upload.delete_token).toBeDefined();
expect(upload.image_url).toBeDefined();

// ensure that image download is working. theres probably a better way to do this
const response = await fetch(upload.url ?? "");
const response = await fetch(upload.image_url ?? "");
const content = await response.text();
expect(content.length).toBeGreaterThan(0);

let community = await resolveBetaCommunity(user);
let post = await createPost(
user,
community.community!.community.id,
upload.url,
upload.image_url,
);
expect(post.post_view.post.url).toBe(upload.url);
expect(post.post_view.post.url).toBe(upload.image_url);
expect(post.post_view.image_details).toBeDefined();

// purge post
Expand All @@ -178,7 +169,7 @@ test("Purge post, linked image removed", async () => {
expect(delete_.success).toBe(true);

// ensure that image is deleted
const response2 = await fetch(upload.url ?? "");
const response2 = await fetch(upload.image_url ?? "");
const content2 = await response2.text();
expect(content2).toBe("");
});
Expand All @@ -200,11 +191,11 @@ test("Images in remote image post are proxied if setting enabled", async () => {
// remote image gets proxied after upload
expect(
post.thumbnail_url?.startsWith(
"http://lemmy-gamma:8561/api/v4/image_proxy?url",
"http://lemmy-gamma:8561/api/v4/image/proxy?url",
),
).toBeTruthy();
expect(
post.body?.startsWith("![](http://lemmy-gamma:8561/api/v4/image_proxy?url"),
post.body?.startsWith("![](http://lemmy-gamma:8561/api/v4/image/proxy?url"),
).toBeTruthy();

// Make sure that it ends with jpg, to be sure its an image
Expand All @@ -223,12 +214,12 @@ test("Images in remote image post are proxied if setting enabled", async () => {

expect(
epsilonPost.thumbnail_url?.startsWith(
"http://lemmy-epsilon:8581/api/v4/image_proxy?url",
"http://lemmy-epsilon:8581/api/v4/image/proxy?url",
),
).toBeTruthy();
expect(
epsilonPost.body?.startsWith(
"![](http://lemmy-epsilon:8581/api/v4/image_proxy?url",
"![](http://lemmy-epsilon:8581/api/v4/image/proxy?url",
),
).toBeTruthy();

Expand All @@ -250,7 +241,7 @@ test("Thumbnail of remote image link is proxied if setting enabled", async () =>
// remote image gets proxied after upload
expect(
post.thumbnail_url?.startsWith(
"http://lemmy-gamma:8561/api/v4/image_proxy?url",
"http://lemmy-gamma:8561/api/v4/image/proxy?url",
),
).toBeTruthy();

Expand All @@ -268,7 +259,7 @@ test("Thumbnail of remote image link is proxied if setting enabled", async () =>

expect(
epsilonPost.thumbnail_url?.startsWith(
"http://lemmy-epsilon:8581/api/v4/image_proxy?url",
"http://lemmy-epsilon:8581/api/v4/image/proxy?url",
),
).toBeTruthy();

Expand All @@ -292,14 +283,14 @@ test("No image proxying if setting is disabled", async () => {
let post = await createPost(
alpha,
community.community_view.community.id,
upload.url,
upload.image_url,
`![](${sampleImage})`,
);
expect(post.post_view.post).toBeDefined();

// remote image doesn't get proxied after upload
expect(
post.post_view.post.url?.startsWith("http://127.0.0.1:8551/pictrs/image/"),
post.post_view.post.url?.startsWith("http://lemmy-beta:8551/api/v4/image/"),
).toBeTruthy();
expect(post.post_view.post.body).toBe(`![](${sampleImage})`);

Expand All @@ -312,7 +303,7 @@ test("No image proxying if setting is disabled", async () => {

// remote image doesn't get proxied after federation
expect(
betaPost.post.url?.startsWith("http://127.0.0.1:8551/pictrs/image/"),
betaPost.post.url?.startsWith("http://lemmy-beta:8551/api/v4/image/"),
).toBeTruthy();
expect(betaPost.post.body).toBe(`![](${sampleImage})`);
// Make sure the alt text got federated
Expand All @@ -334,7 +325,7 @@ test("Make regular post, and give it a custom thumbnail", async () => {
alphaImage,
community.community_view.community.id,
wikipediaUrl,
upload1.url!,
upload1.image_url!,
);

// Wait for the metadata to get fetched, since this is backgrounded now
Expand All @@ -344,7 +335,7 @@ test("Make regular post, and give it a custom thumbnail", async () => {
);
expect(post.post_view.post.url).toBe(wikipediaUrl);
// Make sure it uses custom thumbnail
expect(post.post_view.post.thumbnail_url).toBe(upload1.url);
expect(post.post_view.post.thumbnail_url).toBe(upload1.image_url);
});

test("Create an image post, and make sure a custom thumbnail doesn't overwrite it", async () => {
Expand All @@ -363,14 +354,14 @@ test("Create an image post, and make sure a custom thumbnail doesn't overwrite i
let post = await createPostWithThumbnail(
alphaImage,
community.community_view.community.id,
upload1.url!,
upload2.url!,
upload1.image_url!,
upload2.image_url!,
);
post = await waitUntil(
() => getPost(alphaImage, post.post_view.post.id),
p => p.post_view.post.thumbnail_url != undefined,
);
expect(post.post_view.post.url).toBe(upload1.url);
expect(post.post_view.post.url).toBe(upload1.image_url);
// Make sure the custom thumbnail is ignored
expect(post.post_view.post.thumbnail_url == upload2.url).toBe(false);
expect(post.post_view.post.thumbnail_url == upload2.image_url).toBe(false);
});
1 change: 0 additions & 1 deletion api_tests/src/post.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ import {
listPostReports,
randomString,
registerUser,
getSite,
unfollows,
resolveCommunity,
waitUntil,
Expand Down
11 changes: 5 additions & 6 deletions api_tests/src/shared.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
import {
AdminBlockInstanceParams,
ApproveCommunityPendingFollower,
BlockCommunity,
BlockCommunityResponse,
CommunityId,
CommunityVisibility,
CreatePrivateMessageReport,
DeleteImage,
DeleteImageParams,
EditCommunity,
GetCommunityPendingFollowsCountResponse,
GetReplies,
Expand Down Expand Up @@ -210,7 +209,9 @@ async function allowInstance(api: LemmyHttp, instance: string) {
// Ignore errors from duplicate allows (because setup gets called for each test file)
try {
await api.adminAllowInstance(params);
} catch {}
} catch {
//console.log("Failed to allow instance");
}
}

export async function createPost(
Expand Down Expand Up @@ -715,7 +716,6 @@ export async function saveUserSettingsBio(
export async function saveUserSettingsFederated(
api: LemmyHttp,
): Promise<SuccessResponse> {
let avatar = sampleImage;
let banner = sampleImage;
let bio = "a changed bio";
let form: SaveUserSettings = {
Expand All @@ -724,7 +724,6 @@ export async function saveUserSettingsFederated(
default_post_sort_type: "Hot",
default_listing_type: "All",
interface_language: "",
avatar,
banner,
display_name: "user321",
show_avatars: false,
Expand Down Expand Up @@ -947,7 +946,7 @@ export async function deleteAllImages(api: LemmyHttp) {
Promise.all(
imagesRes.images
.map(image => {
const form: DeleteImage = {
const form: DeleteImageParams = {
token: image.local_image.pictrs_delete_token,
filename: image.local_image.pictrs_alias,
};
Expand Down
Loading