Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(route): sichuan/deyang 政府公开信息 #9453

Merged
merged 66 commits into from
Apr 3, 2022
Merged
Show file tree
Hide file tree
Changes from 65 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
a7fd78b
fead: add 雪球股票评论
zytomorrow May 26, 2021
bd8d281
Merge branch 'master' of https://github.com/DIYgod/RSSHub
zytomorrow May 26, 2021
9d64bbe
style: auto format
actions-user May 26, 2021
60e4e7b
fix: 修复雪球评论转发时未带原文
zytomorrow May 27, 2021
a959c34
Merge branch 'master' of https://github.com/zytomorrow/RSSHub
zytomorrow May 27, 2021
2f47c0b
refactor: 微博个人时间线接入util,支持可读性优化,同时打开emoji表情展示
zytomorrow May 31, 2021
6a8d7ab
route: 雪球评论标题长度自定义
zytomorrow May 31, 2021
bb8a83b
Merge branch 'master' of https://github.com/DIYgod/RSSHub
zytomorrow May 31, 2021
3fd898e
style: auto format
actions-user May 31, 2021
27c76e6
Merge branch 'DIYgod:master' into master
zytomorrow Jun 19, 2021
cec66a6
Merge branch 'DIYgod:master' into master
zytomorrow Jun 28, 2021
71feb80
Merge branch 'DIYgod:master' into master
zytomorrow Jul 5, 2021
41af4d5
fix: 风之动漫域名更换
zytomorrow Jul 5, 2021
7a41f25
fix: 风之漫画新域名适配
zytomorrow Jul 5, 2021
087ee83
refactor: 微博个人时间线接入util,支持可读性优化,同时打开emoji表情展示
zytomorrow Jul 5, 2021
c71e1d6
fix: 雪球股票评论格式调整
zytomorrow Jul 5, 2021
7e664ec
Merge branch 'master' of https://github.com/DIYgod/RSSHub into xueqiu…
zytomorrow Jul 5, 2021
585f2a4
fix: 风之动漫域名更换
zytomorrow Jul 5, 2021
99bd0f6
Merge branch 'master' of https://github.com/DIYgod/RSSHub into weibo_…
zytomorrow Jul 5, 2021
550b712
Merge pull request #60 from zytomorrow/fix-风之动漫域名更换
zytomorrow Jul 5, 2021
d126ce5
Merge pull request #61 from zytomorrow/xueqiu_comments
zytomorrow Jul 5, 2021
f2e1642
Merge branch 'DIYgod:master' into fix-风之动漫域名更换
zytomorrow Jul 6, 2021
93d0e80
Merge pull request #66 from zytomorrow/fix-风之动漫域名更换
zytomorrow Jul 6, 2021
b9882ef
fix: 风之动漫link修复
zytomorrow Jul 18, 2021
74cf039
fix: 雪球股票评论正文使用art模板,日期解析调整
zytomorrow Jul 19, 2021
2a41a15
feat(route): add 德阳考试中心
zytomorrow Jul 19, 2021
1e1ff21
feat(route): add 德阳考试中心
zytomorrow Jul 20, 2021
0b6c84b
fix: 风之动漫获取数量去掉限制
zytomorrow Jul 20, 2021
e04b991
fix: 风之动漫获取数量去掉限制
zytomorrow Jul 20, 2021
947508a
fix: 风之动漫获取数量使用通用参数控制
zytomorrow Jul 20, 2021
7e44bad
fix: 风之动漫获取数量使用通用参数控制
zytomorrow Jul 20, 2021
923928e
Merge pull request #76 from zytomorrow/fix-风之动漫域名更换
zytomorrow Jul 28, 2021
939b768
Merge pull request #77 from zytomorrow/dykszx
zytomorrow Jul 28, 2021
ebf40a4
Merge pull request #78 from zytomorrow/xueqiu_comments
zytomorrow Jul 28, 2021
ccd926d
Merge branch 'master' into weibo_home_timeline_refactor
zytomorrow Jul 28, 2021
cb9dde4
style: auto format
actions-user Jul 28, 2021
a5c2be8
Merge branch 'master' of https://github.com/DIYgod/RSSHub
zytomorrow Oct 9, 2021
a7b3df4
fix: dykszx
zytomorrow Oct 9, 2021
38e3967
Merge branch 'DIYgod:master' into master
zytomorrow Nov 23, 2021
7f09ff5
fix: 德阳考试中心 ssl证书过期
zytomorrow Dec 6, 2021
74780fe
Merge branch 'master' of https://github.com/DIYgod/RSSHub into dykszx
zytomorrow Dec 6, 2021
59e1c87
Merge branch 'dykszx' of https://github.com/zytomorrow/RSSHub
zytomorrow Dec 6, 2021
5cc0232
Merge branch 'master' of https://github.com/DIYgod/RSSHub into dykszx
zytomorrow Dec 6, 2021
5600c71
Merge branch 'master' of https://github.com/DIYgod/RSSHub into dykszx
zytomorrow Dec 6, 2021
b484fb9
Merge branch 'dykszx' of https://github.com/zytomorrow/RSSHub
zytomorrow Dec 6, 2021
7e8c7d9
Merge branch 'DIYgod:master' into master
zytomorrow Jan 4, 2022
061c648
Merge branch 'master' of https://github.com/diyGod/RSSHub
zytomorrow Feb 10, 2022
540a760
tmp1
zytomorrow Feb 10, 2022
b68b70d
tmp2:
zytomorrow Feb 11, 2022
251b7b1
tmp3
zytomorrow Feb 11, 2022
3412f3d
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
b990d03
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
9176b28
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
81ab468
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
b00fe86
Merge pull request #171 from zytomorrow/gov_public_info
zytomorrow Feb 11, 2022
4f67f5e
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
069a06c
Revert "feat(route): add 政府公开信息"
zytomorrow Feb 11, 2022
aa6f366
feat(route): add 政府公开信息
zytomorrow Feb 11, 2022
3f24212
Merge branch 'DIYgod:master' into master
zytomorrow Feb 13, 2022
9cfdb73
fix: 修复德阳政府公开信息
zytomorrow Apr 3, 2022
7022853
fix: 修复德阳政府公开信息
zytomorrow Apr 3, 2022
0c5f167
Merge branch 'master' of https://github.com/DIYgod/RSSHub into fix_ro…
zytomorrow Apr 3, 2022
8e7f5bf
Merge branch 'master' of https://github.com/DIYgod/RSSHub into fix_ro…
zytomorrow Apr 3, 2022
0769000
Merge branch 'master' of https://github.com/DIYgod/RSSHub into fix_ro…
zytomorrow Apr 3, 2022
dcae8a2
Merge branch 'master' of https://github.com/DIYgod/RSSHub into fix_ro…
zytomorrow Apr 3, 2022
8f40ab0
fix: 修复德阳政府公开信息
zytomorrow Apr 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/government.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ pageClass: routes

### 德阳市政府公开信息

<Route author="zytomorrow" example="/gov/sichuan/deyang/govpulicinfo/德阳市市/市人社局" path="/gov/sichuan/deyang/govpulicinfo/:countyName/:institutionName?" :paramsDesc="['区县名。德阳市、绵竹市、广汉市、什邡市、中江县、罗江区、旌阳区、高新区', '单位名称。可直接输入网页显示单位名称']"/>
<Route author="zytomorrow" example="/gov/sichuan/deyang/govpulicinfo/绵竹市" path="/gov/sichuan/deyang/govpulicinfo/:countyName" :paramsDesc="['区县名(**其他区县整改中,暂时只支持`绵竹市`**)。德阳市、绵竹市、广汉市、什邡市、中江县、罗江区、旌阳区、高新区']"/>

## 广东省人民政府

Expand Down
2 changes: 1 addition & 1 deletion lib/v2/gov/maintainer.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ module.exports = {
'/guangdong/tqyb/tfxtq': ['Fatpandac'],
'/guangdong/tqyb/sncsyjxh': ['Fatpandac'],
'/huizhou/zwgk/:category?': ['Fatpandac'],
'/sichuan/deyang/govpulicinfo/:countyName/:institutionName?': ['zytomorrow'],
'/sichuan/deyang/govpulicinfo/:countyName': ['zytomorrow'],
'/mfa/wjdt/:category?': ['nczitzk'],
'/miit/wjfb/:ministry': ['Fatpandac'],
'/miit/yjzj': ['Fatpandac'],
Expand Down
2 changes: 1 addition & 1 deletion lib/v2/gov/radar.js
Original file line number Diff line number Diff line change
Expand Up @@ -296,7 +296,7 @@ module.exports = {
title: '德阳市政府公开信息',
docs: 'https://docs.rsshub.app/government.html#de-yang-shi-fu-ren-min-zheng-zheng-fu',
source: ['/*'],
target: '/sichuan/deyang/govpulicinfo/:countyName/:institutionName?',
target: '/sichuan/deyang/govpulicinfo/:countyName',
},
],
},
Expand Down
2 changes: 1 addition & 1 deletion lib/v2/gov/router.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ module.exports = function (router) {
router.get('/guangdong/tqyb/tfxtq', require('./guangdong/tqyb/tfxtq'));
router.get('/guangdong/tqyb/sncsyjxh', require('./guangdong/tqyb/sncsyjxh'));
router.get('/huizhou/zwgk/:category?', require('./huizhou/zwgk/index'));
router.get('/sichuan/deyang/govpulicinfo/:countyName/:institutionName?', require('./sichuan/deyang/govpulicinfo'));
router.get('/sichuan/deyang/govpulicinfo/:countyName', require('./sichuan/deyang/govpulicinfo'));
router.get('/mfa/wjdt/:category?', require('./mfa/wjdt'));
router.get('/miit/wjfb/:ministry', require('./miit/wjfb'));
router.get('/miit/yjzj', require('./miit/yjzj'));
Expand Down
116 changes: 43 additions & 73 deletions lib/v2/gov/sichuan/deyang/govpulicinfo.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,112 +3,82 @@ const cheerio = require('cheerio');
const { parseDate } = require('@/utils/parse-date');
const { art } = require('@/utils/render');
const path = require('path');
const timezone = require('@/utils/timezone');

const rootUrl = 'http://xxgk.deyang.gov.cn/xxgkml2020';

// 地区名称对照表
const nameDict = {
德阳市: 'dys',
绵竹市: 'mzs',
什邡市: 'sfs',
旌阳区: 'jyq',
罗江区: 'ljx',
广汉市: 'ghs',
中江县: 'zjx',
高新区: 'gxq',
// 各地区url信息
const basicInfoDict = {
绵竹市: {
rootUrl: 'https://www.mz.gov.cn',
infoUrl: 'https://www.mz.gov.cn/info/iList.jsp?tm_id=1805&cur_page=',
},
};

const getInstitutionId = async (ctx, county) => {
const url = `${rootUrl}/ptlj.jsp?regionName=${county}`;

return await ctx.cache.tryGet(`${county}InstitutionId`, async () => {
const response = await got.get(url);
const getInfoUrlList = async (countyName) => {
const url = basicInfoDict[countyName].infoUrl;
const infoUrlList = [];
const expectToday = parseDate(timezone(new Date(), +8));
for (let pageNum = 1; ; pageNum++) {
// eslint-disable-next-line no-await-in-loop
const response = await got.get(`${url}${pageNum}`);
zytomorrow marked this conversation as resolved.
Show resolved Hide resolved
const $ = cheerio.load(response.data);
const dataList = $('#details_content > div > div > div > div:nth-child(4) > ul > li > a');
const _tmp = {};
for (let i = 0; i < dataList.length; i++) {
_tmp[$(dataList[i]).html()] = parseInt($(dataList[i]).attr('href').split('deptId=')[1]);
const infoList = $('#list_content > ul > li');
// 判断当前页是否有数据
if (infoList.length !== 0) {
for (let infoIdx = 0; infoIdx < infoList.length; infoIdx++) {
const date = parseDate($('span', infoList[infoIdx]).html());
// 判断当前信息日期是否是今日
if (date.getUTCFullYear() === expectToday.getUTCFullYear() && date.getUTCMonth() === expectToday.getUTCMonth() && date.getUTCDay() === expectToday.getUTCDay()) {
infoUrlList.push(`${basicInfoDict[countyName].rootUrl}${$('a', infoList[infoIdx]).attr('href')}`);
} else {
return infoUrlList;
}
}
} else {
return infoUrlList;
}
return _tmp;
});
};

const getInfoUrlList = async (county, institutionId) => {
const url = `${rootUrl}/gklist_iframe.jsp?deptId=${institutionId}&regionName=${county}&pageSize=15`;
const response = await got.get(url);
const $ = cheerio.load(response.data);
const pageNum = parseInt($('#list_content > div > span:nth-child(3)').html().match(/\d*/g)[1]);
const pageUrlList = [];

// 此处建议限制最大pageNum,最大5页,单页15条
for (let i = 1; i <= Math.min(pageNum, 5); i++) {
pageUrlList.push(`${rootUrl}/gklist_iframe.jsp?deptId=${institutionId}&regionName=${county}&pageSize=15$pageIndex=${i}`);
}
const _tmpList = await Promise.all(
pageUrlList.map(async (url) => {
const response = await got.get(url);
const $ = cheerio.load(response.data);
const InfoList = $('#list_content > ul > li > a');
return InfoList.map((item) => `${rootUrl}/${$(InfoList[item]).attr('href')}`);
})
);
let infoUrlList = [];
for (let i = 0; i < _tmpList.length; i++) {
infoUrlList = infoUrlList.concat(_tmpList[i].toArray());
}
return infoUrlList;
};

const getInfoContent = async (ctx, url) => {
const infoId = url.split('id=')[1].split('&type')[0];
return await ctx.cache.tryGet(`govPublicInfo${infoId}`, async () => {
// 获取信息正文内容
const getInfoContent = async (ctx, url, countyName) =>
await ctx.cache.tryGet(url, async () => {
const response = await got.get(url);
const $ = cheerio.load(response.data);
const fileNodes = $('#symbol > div:nth-child(3) > div > a');
const fileNodes = $('#symbol > div:nth-child(4) > div > span > a');
const fileList = [];
for (let i = 0; i < fileNodes.length; i++) {
fileList.push({
name: $(fileNodes[i]).text(),
url: `${rootUrl}/${$(fileNodes[i]).attr('href')}`,
url: `${basicInfoDict[countyName].rootUrl}/${$(fileNodes[i]).attr('href')}`,
});
}
const rawDate = $('#symbol > div:nth-child(1) > div:nth-child(3)').text().split('\n')[1].trim();
const rawDate = $('#symbol > div:nth-child(1) > div:nth-child(3)').text().split('')[1].trim();
return {
title: $('#headline').text(),
id: infoId,
infoNum: $('#symbol > div:nth-child(1) > div:nth-child(2) > span').text().split('\n')[1].trim(),
title: $('#main').text().trim(),
id: $('#symbol > div:nth-child(1) > div:nth-child(1)').text().split(':')[1].trim(),
infoNum: $('#symbol > div:nth-child(1) > div:nth-child(2) > span').text().split('')[1].trim(),
pubDate: parseDate(rawDate),
date: rawDate,
keyWord: $('#symbol > div:nth-child(2) > div:nth-child(1)').text().slice(8),
source: $('#symbol > div:nth-child(2) > div:nth-child(3)').text().slice(5),
content: $('#details_content > div.content').html(),
keyWord: $('#symbol > div:nth-child(2) > div:nth-child(3)').text().split(':')[1].trim(),
source: $('#symbol > div:nth-child(2) > div:nth-child(2)').text().split(':')[1].trim(),
content: $('#container > div.ewb-white > div.ewb-article-detail').html(),
file: fileList,
link: url,
};
});
};

module.exports = async (ctx) => {
const countyName = ctx.params.countyName;
const county = nameDict[countyName];
const institutionName = ctx.params.institutionName || '';
let institutionId = 0;
if (institutionName) {
const institutionDict = await getInstitutionId(ctx, county);
institutionId = institutionDict[institutionName];
}

const infoUrlList = await getInfoUrlList(county, institutionId);
const infoUrlList = await getInfoUrlList(countyName);
const items = await Promise.all(infoUrlList.map(async (item) => await getInfoContent(ctx, item)));

ctx.state.data = {
title: `政府公开信息 - ${countyName} ${institutionName}`,
link: `${rootUrl}/gklist_iframe.jsp?deptId=${institutionId}&regionName=${county}`,
title: `政府公开信息 - ${countyName}`,
link: basicInfoDict[countyName].infoUrl,
item: items.map((item) => ({
title: item.title,
description: art(path.resolve(__dirname, './templates/govPublicInfo.art'), { item }),
link: item.link,
guid: item.id,
pubDate: item.pubDate,
})),
};
Expand Down
1 change: 1 addition & 0 deletions lib/v2/gov/sichuan/deyang/templates/govPublicInfo.art
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
</tbody>
</table>

<hr/>
<div>
{{@ item.content}}
</div>
Expand Down