web-crawler

所属分类:数据采集/爬虫
开发工具:JavaScript
文件大小:0KB
下载次数:0
上传日期:2023-09-12 19:21:12
上 传 者sh-1993
说明:  一个网络爬虫,用于从新闻网站的首页提取新闻文章,并自动将内容从德语翻译为英语,
(A web crawler for pulling news articles from the front page of a news site, and auto translating the content from German to English,)

文件列表:
.DS_Store (6148, 2023-09-12)
.editorconfig (258, 2023-09-12)
.env.example (991, 2023-09-12)
.styleci.yml (194, 2023-09-12)
app/ (0, 2023-09-12)
app/ApiClients/ (0, 2023-09-12)
app/ApiClients/GuzzleClient.php (0, 2023-09-12)
app/Console/ (0, 2023-09-12)
app/Console/Commands/ (0, 2023-09-12)
app/Console/Commands/WebCrawlerCommand.php (998, 2023-09-12)
app/Console/Kernel.php (681, 2023-09-12)
app/Contracts/ (0, 2023-09-12)
app/Contracts/ClientInterface.php (104, 2023-09-12)
app/Contracts/ReaderInterface.php (1072, 2023-09-12)
app/Exceptions/ (0, 2023-09-12)
app/Exceptions/Handler.php (824, 2023-09-12)
app/Http/ (0, 2023-09-12)
app/Http/Controllers/ (0, 2023-09-12)
app/Http/Controllers/ArticleController.php (355, 2023-09-12)
app/Http/Controllers/Controller.php (361, 2023-09-12)
app/Http/Kernel.php (2580, 2023-09-12)
app/Http/Middleware/ (0, 2023-09-12)
app/Http/Middleware/Authenticate.php (469, 2023-09-12)
app/Http/Middleware/EncryptCookies.php (307, 2023-09-12)
app/Http/Middleware/PreventRequestsDuringMaintenance.php (366, 2023-09-12)
app/Http/Middleware/RedirectIfAuthenticated.php (877, 2023-09-12)
app/Http/Middleware/TrimStrings.php (381, 2023-09-12)
app/Http/Middleware/TrustHosts.php (372, 2023-09-12)
app/Http/Middleware/TrustProxies.php (649, 2023-09-12)
app/Http/Middleware/VerifyCsrfToken.php (320, 2023-09-12)
app/Jobs/ (0, 2023-09-12)
app/Jobs/ArticleCrawlerJob.php (1207, 2023-09-12)
app/Models/ (0, 2023-09-12)
app/Models/Article.php (208, 2023-09-12)
app/Models/User.php (913, 2023-09-12)
app/Providers/ (0, 2023-09-12)
app/Providers/AppServiceProvider.php (635, 2023-09-12)
... ...

A simple web crawler, with the ability to pull and translate text to English. ## Technology Stack and Tools - Laravel - Google Translate - Vue JS - MySQL - CSS Bootsrap - Axios HTTP Client - XML ## Set up Instruction - Clone the repo - CD into cloned directory and run `touch .env` to create a new .env file - Copy all contents in .env.example into newly created .env file - Install composer `composer install` - Install npm modules `npm install` - Generate a new key with `php artisan key:generate` - Open PHPMyAdmin or whichever tool you use and create a new mysql database - Open .env and add db credentials - Clear config cache with `php artisan cache:clear` - Run migrations with `php artisan migrate` ## Web crawling - Run `php artisan web:crawl` ## Demo Link - https://webcrawler.monamoxie.com/

## License This software is open-sourced software and licensed under the [MIT license](https://opensource.org/licenses/MIT).

近期下载者

相关文件


收藏者