Metrology Information Data Push System Based on Crawler and Deep Learning

JIANG Haiyang; TIAN Feng; ZHI Feng; YANG Yifan

doi:10.12338/j.issn.2096-9015.2020.9030

Volume 65 Issue 9

Sep. 2021

Turn off MathJax

Article Contents

Article Navigation > Metrology Science and Technology > 2021 > 65(9): 35-39

JIANG Haiyang, TIAN Feng, ZHI Feng, YANG Yifan. Metrology Information Data Push System Based on Crawler and Deep Learning[J]. Metrology Science and Technology, 2021, 65(9): 35-39. doi: 10.12338/j.issn.2096-9015.2020.9030

Citation:

JIANG Haiyang, TIAN Feng, ZHI Feng, YANG Yifan. Metrology Information Data Push System Based on Crawler and Deep Learning[J]. Metrology Science and Technology, 2021, 65(9): 35-39. doi: 10.12338/j.issn.2096-9015.2020.9030

Citation:

PDF( 691 KB)

Metrology Information Data Push System Based on Crawler and Deep Learning

doi: 10.12338/j.issn.2096-9015.2020.9030

National Institute of Metrology, Beijing 100029, China

Available Online: 2021-04-21
Publish Date: 2021-09-01

Abstract

Abstract

The National Metrology Data Center has developed a metrology information data push system. The system first uses the web crawler technology to capture the latest metrology information data from authoritative metrology websites at home and abroad. Then the data is integrated to a three-dimensional topic map after being preprocessed by AI and manually curated, which is further published to the website and pushed to the corresponding subscribers with corresponding subject terms. Researchers can subscribe to the information according to the three dimensional topic maps or custom subject terms to obtain the latest metrology information data in real time. The system has been deployed on the website of the National Metrology Data Center, and the actual operation is good, which verifies the feasibility and development potential of the program.
- metrology information data,
- web crawler technology,
- deep learning,
- topic maps,
- big data

FullText(HTML)

References(12)

References

[1]	李晓萌, 田方, 阳彩军, 等. 我国计量科研领域近20年科研产出对比分析[J]. 计量技术, 2019(9): 62-66.
[2]	郭丽蓉. 基于Python的网络爬虫程序设计[J]. 电子技术与软件工程, 2017(23): 248-249.
[3]	Kevin. 网络爬虫技术原理[J]. 计算机与网络, 2018, 44(10): 38-40.
[4]	徐远超, 刘江华, 刘丽珍, 等. 基于Web的网络爬虫的设计与实现[J]. 微计算机信息, 2007(21): 119-121.
[5]	Chakrabarti S, Berg M V D, Dom B. Focused crawling: a new approach to topic-specific Web resource discovery[J]. Computer Networks, 1999, 31(11-16): 1623-1640. doi: 10.1016/S1389-1286(99)00052-3
[6]	Aggarwal C C, GARAWI A F, YU P S. Intelligent Crawling on the World Wide Web with Arbitrary Predicates[C]. 10th International Conference on World Wide Web. New York: ACM, 2001: 96-105.
[7]	EHRIG M, MAEDCHE A. Ontology-focused Crawling of Web Documents[C]. 2003 ACM Symposium on Applied Computing. New York: ACM, 2003: 1174-1178.
[8]	张环, 刘乃文, 段会川. 基于T-Graph算法的主题爬虫研究[J]. 计算机工程与设计, 2014, 35(9): 3014-3017.
[9]	Manku G S, Jain A, Das S A. Detecting near-duplicates for web crawling[C]. International Conference on World Wide Web. New York: ACM, 2007: 141-149.
[10]	Biezunski M, Bryan M, Steven R. ISO/IEC Standard 13250: Topic Maps[S]. International Organization for Standardization, 1999.
[11]	田海燕. 基于主题地图的文献组织方法研究[D]. 大连: 大连理工大学, 2006.
[12]	Salton G, Buckley C. Term-weighting approaches in automatic text retrieval[J]. Information Processing & Management, 1988, 24(5): 513-523.