Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from diygod:master #1019

Merged
merged 3 commits into from
Aug 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 26 additions & 22 deletions docs/new-media.md
Original file line number Diff line number Diff line change
Expand Up @@ -481,7 +481,7 @@ pageClass: routes

### Headline

<Route author="nczitzk" example="/fashionnetwork/headline" path="/fashionnetwork/headline/:country?" :paramsDesc="['国家,见下表,默认为 `ww`']">
<Route author="nczitzk" example="/fashionnetwork/headline" path="/fashionnetwork/headline/:country?" :paramsDesc="['国家,见下表,默认为 `ww`']">
</Route>

### News
Expand Down Expand Up @@ -904,7 +904,7 @@ Edition

::: tip 提示

在 Matataki 发表的文章会上传到星际文件系统(IPFS),永久保存。即使站内文章因为各种原因消失,用 RSS 获取过带 IPFS 连接的 Feed Item 的话,还是可以从 RSS 阅读器找回文章的。
在 Matataki 发表的文章会上传到星际文件系统(IPFS),永久保存。即使站内文章因为各种原因消失,用 RSS 获取过带 IPFS 连接的 Feed Item 的话,还是可以从 RSS 阅读器找回文章的。
IPFS 网关有可能失效,那时候换成其他网关。

:::
Expand Down Expand Up @@ -1032,6 +1032,10 @@ IPFS 网关有可能失效,那时候换成其他网关。

<Route author="ncziztk" example="/odaily/newsflash" path="/odaily/newsflash"/>

### 搜索快讯

<Route author="snowraincloud" example="/odaily/search/news/Lens%20Protocol" path="/odaily/search/news/:keyword" :paramsDesc="['搜索关键字']"/>

### 文章

<Route author="ncziztk" example="/odaily" path="/odaily/:id?" :paramsDesc="['id,见下表,默认为最新']">
Expand Down Expand Up @@ -3085,50 +3089,50 @@ column 为 third 时可选的 category:

- 人工智能

| 学术 | 开发者 |
| -------- | -------- |
| 学术 | 开发者 |
| -------- | -------- |
| academic | yanxishe |

- 数智化

| 零售数智化 | 金融数智化 | 工业数智化 | 医疗数智化 | 城市数智化 |
| ---------- | ---------- | ---------- | ---------- | ----------- |
| 零售数智化 | 金融数智化 | 工业数智化 | 医疗数智化 | 城市数智化 |
| ---------- | ---------- | ---------- | ---------- | ----------- |
| redigital | findigital | mandigital | medigital | citydigital |

- 金融科技

| 科技巨头 | 银行 AI | 金融云 | 风控与安全 |
| -------- | ------- | ------------ | ------------ |
| 科技巨头 | 银行 AI | 金融云 | 风控与安全 |
| -------- | ------- | ------------ | ------------ |
| BigTech | bank | FinanceCloud | DataSecurity |

- 医疗科技

| 医疗 AI | 投融资 | 医疗器械 | 互联网医疗 | 生物医药 | 健康险 |
| -------- | ------ | -------- | ---------------- | ------------ | ------------ |
| 医疗 AI | 投融资 | 医疗器械 | 互联网医疗 | 生物医药 | 健康险 |
| -------- | ------ | -------- | ---------------- | ------------ | ------------ |
| healthai | touzi | qixie | hulianwangyiliao | shengwuyiyao | jiankangxian |

- 芯片

| 材料设备 | 芯片设计 | 晶圆代工 | 封装测试 |
| --------- | ---------- | ------------- | --------- |
| 材料设备 | 芯片设计 | 晶圆代工 | 封装测试 |
| --------- | ---------- | ------------- | --------- |
| materials | chipdesign | manufacturing | packaging |

- 智慧城市

| 智慧安防 | 智慧教育 | 智慧交通 | 智慧社区 | 智慧零售 | 智慧政务 | 智慧地产 |
| ------------- | -------------- | ------------------- | -------------- | -------------- | --------------- | -------- |
| 智慧安防 | 智慧教育 | 智慧交通 | 智慧社区 | 智慧零售 | 智慧政务 | 智慧地产 |
| ------------- | -------------- | ------------------- | -------------- | -------------- | --------------- | -------- |
| smartsecurity | smarteducation | smarttransportation | smartcommunity | smartretailing | smartgovernment | proptech |

- 工业互联网

| 工业软件 | 工业安全 | 5G 工业互联网 | 工业转型实践 |
| ---------- | -------- | ------------- | ------------ |
| 工业软件 | 工业安全 | 5G 工业互联网 | 工业转型实践 |
| ---------- | -------- | ------------- | ------------ |
| gysoftware | gysafety | 5ggy | gypratice |

- AIoT

| 物联网 | 智能硬件 | 机器人 | 智能家居 |
| ------ | -------- | ------ | --------- |
| 物联网 | 智能硬件 | 机器人 | 智能家居 |
| ------ | -------- | ------ | --------- |
| 5G | arvr | robot | smarthome |

</Route>
Expand Down Expand Up @@ -3580,8 +3584,8 @@ column 为 third 时可选的 category:

- `fulltext`,全文输出,例如:`/pingwest/tag/ChinaJoy/1/fulltext`

::: tip 提示
该路由一次最多显示 30 条文章
::: tip 提示
该路由一次最多显示 30 条文章
:::

</Route>
Expand Down Expand Up @@ -4172,7 +4176,7 @@ column 为 third 时可选的 category:

<Route author="mjysci" example="/163/dy2/T1555591616739" path="/163/dy2/:id" :paramsDesc="['id,该网易号主页网址最后一项html的文件名']" anticrawler="1"/>

优先使用方法一,若是网易号搜索页面搜不到的小众网易号(文章页面不含`data-wemediaid`)则可使用此法。
优先使用方法一,若是网易号搜索页面搜不到的小众网易号(文章页面不含`data-wemediaid`)则可使用此法。
触发反爬会只抓取到标题,建议自建。

## 网易新闻
Expand Down Expand Up @@ -4920,7 +4924,7 @@ column 为 third 时可选的 category:

### 军事 - 军事新闻

<Route author="jiaaoMario" example="/china/news/military" path="/china/news/military">
<Route author="jiaaoMario" example="/china/news/military" path="/china/news/military">
</Route>

### 时事新闻
Expand Down
64 changes: 44 additions & 20 deletions lib/v2/finviz/news.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,43 +3,67 @@ const cheerio = require('cheerio');
const timezone = require('@/utils/timezone');
const { parseDate } = require('@/utils/parse-date');

const categories = {
news: 0,
blogs: 1,
};

module.exports = async (ctx) => {
const category = ctx.params.category ?? 'news';
const { category = 'News' } = ctx.params;
const limit = ctx.query.limit ? parseInt(ctx.query.limit, 10) : 200;

if (!categories.hasOwnProperty(category.toLowerCase())) {
throw Error(`No category '${category}'.`);
}

const rootUrl = 'https://finviz.com';
const currentUrl = `${rootUrl}/news.ashx`;
const currentUrl = new URL('news.ashx', rootUrl).href;

const response = await got({
method: 'get',
url: currentUrl,
});
const { data: response } = await got(currentUrl);

const $ = cheerio.load(response.data);
const $ = cheerio.load(response);

const items = $('.table-fixed')
.eq(category === 'blog' ? 1 : 0)
.find('.nn')
const items = $('table.table-fixed')
.eq(categories[category.toLowerCase()])
.find('tr.nn')
.slice(0, limit)
.toArray()
.map((item) => {
item = $(item);

const a = item.find('.nn-tab-link');
const a = item.find('a.nn-tab-link');

const descriptionMatches = a
.parent()
.prop('data-boxover')
?.match(/<td class='news_tooltip-tab'>(.*?)<\/td>/);
const authorMatches = item
.find('use')
.first()
.prop('href')
?.match(/#(.*?)-(light|dark)/);

return {
title: a.text(),
link: a.attr('href'),
pubDate: timezone(parseDate(item.find('.nn-date').text(), ['MMM-DD', 'HH:mmA']), -4),
description:
item
.find('td[data-boxover]')
.attr('data-boxover')
?.match(/<td class='news_tooltip-tab'>([\s\S]*)<\/td>/)[1] ?? '',
link: a.prop('href'),
description: descriptionMatches ? descriptionMatches[1] : undefined,
author: authorMatches ? authorMatches[1].replace(/-/g, ' ') : 'finviz',
pubDate: timezone(parseDate(item.find('td.nn-date').text(), ['HH:mmA', 'MMM-DD']), -4),
};
});
})
.filter((item) => item.title);

const icon = $('link[rel="icon"]').prop('href');

ctx.state.data = {
item: items,
title: `finviz - ${category}`,
link: currentUrl,
item: items,
description: $('meta[name="description"]').prop('content'),
language: 'en-US',
image: new URL($('a.logo svg use').first().prop('href'), rootUrl).href,
icon,
logo: icon,
subtitle: $('title').text(),
};
};
1 change: 1 addition & 0 deletions lib/v2/odaily/maintainer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
module.exports = {
'/activity': ['nczitzk'],
'/newsflash': ['nczitzk'],
'/search/news/:keyword': ['snowraincloud'],
'/user/:id': ['nczitzk'],
'/:id?': ['nczitzk'],
};
96 changes: 38 additions & 58 deletions lib/v2/odaily/radar.js
Original file line number Diff line number Diff line change
@@ -1,60 +1,40 @@
const odaily = {
_name: 'Odaily 星球日报',
'.': [
{
title: '快讯',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-kuai-xun',
source: ['/newsflash', '/'],
target: '/odaily/newsflash',
},
{
title: '搜索快讯',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-sou-suo-kuai-xun',
source: ['/search/:keyword'],
target: '/odaily/search/news/:keyword',
},
{
title: '文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-wen-zhang',
source: ['/'],
target: '/odaily/:id?',
},
{
title: '用户文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-yong-hu-wen-zhang',
source: ['/user/:id', '/'],
target: '/odaily/user/:id',
},
{
title: '活动',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-huo-dong',
source: ['/activityPage', '/'],
target: '/odaily/activity',
},
],
};

module.exports = {
'odaily.news': {
_name: 'Odaily 星球日报',
'.': [
{
title: '快讯',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-kuai-xun',
source: ['/newsflash', '/'],
target: '/odaily/newsflash',
},
{
title: '文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-wen-zhang',
source: ['/'],
target: '/odaily/:id?',
},
{
title: '用户文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-yong-hu-wen-zhang',
source: ['/user/:id', '/'],
target: '/odaily/user/:id',
},
{
title: '活动',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-huo-dong',
source: ['/activityPage', '/'],
target: '/odaily/activity',
},
],
},
'0daily.com': {
_name: 'Odaily 星球日报',
'.': [
{
title: '快讯',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-kuai-xun',
source: ['/newsflash', '/'],
target: '/odaily/newsflash',
},
{
title: '文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-wen-zhang',
source: ['/'],
target: '/odaily/:id?',
},
{
title: '用户文章',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-yong-hu-wen-zhang',
source: ['/user/:id', '/'],
target: '/odaily/user/:id',
},
{
title: '活动',
docs: 'https://docs.rsshub.app/new-media.html#odaily-xing-qiu-ri-bao-huo-dong',
source: ['/activityPage', '/'],
target: '/odaily/activity',
},
],
},
'odaily.news': odaily,
'0daily.com': odaily,
};
1 change: 1 addition & 0 deletions lib/v2/odaily/router.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
module.exports = function (router) {
router.get('/activity', require('./activity'));
router.get('/newsflash', require('./newsflash'));
router.get('/search/news/:keyword', require('./search_news'));
router.get('/user/:id', require('./user'));
router.get('/:id?', require('./post'));
};
26 changes: 26 additions & 0 deletions lib/v2/odaily/search_news.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
const got = require('@/utils/got');
const timezone = require('@/utils/timezone');
const { parseDate } = require('@/utils/parse-date');
const { rootUrl } = require('./utils');

module.exports = async (ctx) => {
const currentUrl = `${rootUrl}/api/pp/api/search/entity-search?per_page=${ctx.query.limit ?? 25}&keyword=${ctx.params.keyword}&entity_type=newsflash`;

const response = await got({
method: 'get',
url: currentUrl,
});

const items = response.data.data.items.map((item) => ({
title: item.title,
link: item.news_url,
pubDate: timezone(parseDate(item.published_at), +8),
description: `<p>${item.description}</p>`,
}));

ctx.state.data = {
title: '快讯 - Odaily星球日报',
link: `${rootUrl}/search/${ctx.params.keyword}`,
item: items,
};
};
3 changes: 1 addition & 2 deletions lib/v2/telegram/radar.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ module.exports = {
docs: 'https://docs.rsshub.app/social-media.html#telegram',
source: '/:username',
target: (params, url, document) => {
const isChannel = document && document.querySelector('.tgme_action_button_label');
if (isChannel) {
if (document?.querySelector('a[href^="/s/"]')) {
return '/telegram/channel/:username';
}
},
Expand Down