NewsClassification-Celery-Gensim-Spacy
所属分类:自然语言处理
开发工具:Jupyter Notebook
文件大小:0KB
下载次数:0
上传日期:2024-01-31 05:36:23
上 传 者:
sh-1993
说明: 新闻分类芹菜Gensim Spacy
(NewsClassification Celery Gensim Spacy)
文件列表:
News_Ingestion Task/
# Documentation
## Files and Functions
- database_storage.py (Responsible for storage of parsed data to database without dupicates)
- feed_parser.py (Responsible for parsing links async using celery and RabbitMQ)
- news_processing.py (Responsible for classifying news into categories using gensim and spacy)
- main.py (main file for bringing code together)
- config files have configs related to celery and postgresql database
## Running
RabbitMQ and postgresql service must be running in background
Runnign celery worker on terminal
```
celery -A feed_parser worker -P gevent --loglevel=info
```
Running Program
```sh
python main.py
```
近期下载者:
相关文件:
收藏者: