Mission_to_Mars
所属分类:交通/航空行业
开发工具:Jupyter Notebook
文件大小:3625KB
下载次数:0
上传日期:2018-05-11 03:49:53
上 传 者:
sh-1993
说明: 抓取美国国家航空航天局网站以获取火星的最新新闻、数据和图像。
(Scrape NASA website to get updated news, data, and images of Mars.)
文件列表:
.DS_Store (10244, 2018-05-11)
.ipynb_checkpoints (0, 2018-05-11)
.ipynb_checkpoints\mission_to_mars-checkpoint.ipynb (15652, 2018-05-11)
Images (0, 2018-05-11)
Images\final_app_part1.png (1783414, 2018-05-11)
Images\final_app_part2.png (1847066, 2018-05-11)
Images\mission_to_mars.jpg (167381, 2018-05-11)
Images\requirements.txt (32, 2018-05-11)
__pycache__ (0, 2018-05-11)
__pycache__\scrape_mars.cpython-36.pyc (2596, 2018-05-11)
app.py (551, 2018-05-11)
index.html (3200, 2018-05-11)
mission_to_mars.ipynb (15652, 2018-05-11)
scrape_mars.py (3840, 2018-05-11)
table.html (1099, 2018-05-11)
templates (0, 2018-05-11)
templates\index.html (3300, 2018-05-11)
# Mission to Mars
![mission_to_mars](https://github.com/seaaaaany/Mission_to_Mars/blob/master/Images/mission_to_mars.jpg)
In this assignment, I built a web application that scrapes various websites for data related to the Mission to Mars and displays the information in a single HTML page.
## Step 1 - Scraping
I completed initial scraping using Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter.
* Created a Jupyter Notebook file called `mission_to_mars.ipynb` and use this to complete all of your scraping and analysis tasks. The following outlines what you need to scrape.
### NASA Mars News
* Scrape the [NASA Mars News Site](https://github.com/seaaaaany/Mission_to_Mars/blob/master/https://mars.nasa.gov/news/) and collect the latest News Title and Paragragh Text. Assign the text to variables that you can reference later.
```python
# Example:
news_title = "NASA's First Deep-Space CubeSats Say: 'Polo!'"
news_p = "MarCO is a pair of tiny spacecraft that launched with NASA's InSight lander today."
```
### JPL Mars Space Images - Featured Image
* Visit the url for JPL's Featured Space Image [here](https://github.com/seaaaaany/Mission_to_Mars/blob/master/https://www.jpl.nasa.gov/spaceimages/?search=&category=Mars).
* Use splinter to navigate the site and find the image url for the current Featured Mars Image and assign the url string to a variable called `featured_image_url`.
* Make sure to find the image url to the full size `.jpg` image.
* Make sure to save a complete url string for this image.
```python
# Example:
featured_image_url = 'https://www.jpl.nasa.gov/spaceimages/images/largesize/PIA16225_hires.jpg'
```
### Mars Weather
* Visit the Mars Weather twitter account [here](https://github.com/seaaaaany/Mission_to_Mars/blob/master/https://twitter.com/marswxreport?lang=en) and scrape the latest Mars weather tweet from the page. Save the tweet text for the weather report as a variable called `mars_weather`.
```python
# Example:
mars_weather = 'Sol 1801 (Aug 30, 2017), Sunny, high -21C/-5F, low -80C/-112F, pressure at 8.82 hPa, daylight 06:09-17:55'
```
### Mars Facts
* Visit the Mars Facts webpage [here](https://github.com/seaaaaany/Mission_to_Mars/blob/master/http://space-facts.com/mars/) and use Pandas to scrape the table containing facts about the planet including Diameter, Mass, etc.
* Use Pandas to convert the data to a HTML table string.
### Mars Hemisperes
* Visit the USGS Astrogeology site [here](https://github.com/seaaaaany/Mission_to_Mars/blob/master/https://astrogeology.usgs.gov/search/results?q=hemisphere+enhanced&k1=target&v1=Mars) to obtain high resolution images for each of Mar's hemispheres.
* You will need to click each of the links to the hemispheres in order to find the image url to the full resolution image.
* Save both the image url string for the full resolution hemipshere image, and the Hemisphere title containing the hemisphere name. Use a Python dictionary to store the data using the keys `img_url` and `title`.
* Append the dictionary with the image url string and the hemisphere title to a list. This list will contain one dictionary for each hemisphere.
```python
# Example:
hemisphere_image_urls = [
{"title": "Valles Marineris Hemisphere", "img_url": "..."},
{"title": "Cerberus Hemisphere", "img_url": "..."},
{"title": "Schiaparelli Hemisphere", "img_url": "..."},
{"title": "Syrtis Major Hemisphere", "img_url": "..."},
]
```
---
## Step 2 - MongoDB and Flask Application
Use MongoDB with Flask templating to create a new HTML page that displays all of the information that was scraped from the URLs above.
* Start by converting your Jupyter notebook into a Python script called `scrape_mars.py` with a function called `scrape` that will execute all of your scraping code from above and return one Python dictionary containing all of the scraped data.
* Next, create a route called `/scrape` that will import your `scrape_mars.py` script and call your `scrape` function.
* Store the return value in Mongo as a Python dictionary.
* Create a root route `/` that will query your Mongo database and pass the mars data into an HTML template to display the data.
* Create a template HTML file called `index.html` that will take the mars data dictionary and display all of the data in the appropriate HTML elements. Use the following as a guide for what the final product should look like, but feel free to create your own design.
![final_app_part1.png](https://github.com/seaaaaany/Mission_to_Mars/blob/master/Images/final_app_part1.png)
![final_app_part2.png](https://github.com/seaaaaany/Mission_to_Mars/blob/master/Images/final_app_part2.png)
---
## Hints
* Use splinter to navigate the sites when needed and BeautifulSoup to help find and parse out the necessary data.
* Use Pymongo for CRUD applications for your database. For this homework, you can simply overwrite the existing document each time the `/scrape` url is visited and new data is obtained.
* Use Bootstrap to structure your HTML template.
## Copyright
Coding Boot Camp 2017. All Rights Reserved.
近期下载者:
相关文件:
收藏者: