Abben
Apr 22, 2020

地球观测/地理空间相关资源大列表

1 comment
Abben
Apr 22, 2020

Python processing of imagery non deep learning


  • Povetry predition using satellite imagery - Poverty Prediction by Combination of Satellite Imagery

  • Remote Sensing indicies calc - Calculate spectral remote sensing indices from satellite imagery

  • Satellite imagery in Pakistan - This repository contains a study how we can examine the vegetation cover of a region with the help of satellite data. The notebook in this repository aims to familiarise with the concept of satellite imagery data and how it can be analyzed to investigate real-world environmental and humanitarian challenges.

  • s3 tools - A collection of sentinel 3 processing tools

  • eumetsat -python - Shows how to read and plot satellite data from EUMETSAT NETCDF files

  • unidata on GOES-16 - This notebook shows how to make a true color image from the GOES-16 Advanced Baseline Imager (ABI) level 2 data. We will plot the image with matplotlib and Cartopy.

  • esa_sentinel - ESA Sentinel Search & Download API

  • MetPy - MetPy is a collection of tools in Python for reading, visualizing and performing calculations with weather data.

  • MetPy docs


  • EarthPy - A package built to support working with spatial data using open source python. docs

  • RasterFrames / pyrasterframes - brings together Earth-observation (EO) data access, cloud computing, and DataFrame-based data science. docs

Resources for R

R is not my area of expertise so this section is lighter than I'd like, plus I'd love to know what is a useful resource


  • GDAL Cubes - Earth Observation Data Cubes from Satellite Image Collections. Also here on github

  • Image Classification with RandomForests in R - The goal of this post is to demonstrate the ability of R to classify multispectral imagery using RandomForests algorithms.

  • R code for ML in Sat imagery - # Random Forest image classification Adapted from stackoverflow.

  • whiteboxR - An R frontend of the advanced geospatial data analysis platform - whitebox-tools.

  • RasterVIS - Methods for enhanced visualization and interaction with raster data. It implements visualization methods for quantitative data and categorical data, both for univariate and multivariate rasters. It also provides methods to display spatiotemporal rasters, and vector fields.

  • Landsat - Processing of Landsat or other multispectral satellite imagery. Includes relative normalization, image-based radiometric correction, and topographic correction options.

  • rnoaa - R interface to many NOAA data APIs

  • MODISTools - Interface to the MODIS Land Products Subsets Web Services Docs

Languages other than Python



  • GDAL with GoLang - Go (golang) wrapper for GDAL, the Geospatial Data Abstraction Library

  • C++ gdalcubes - Earth observation data cubes from GDAL image collections

  • RSGLib - The remote sensing and GIS software library (RSGISLib) is a set of C++ libraries and commands for the processing of spatial data (raster and vector). Functionality is via Python interface though

  • WhiteBox with Java - An open-source GIS and remote sensing package

  • Perl extension for GDAL - Geo::GDAL - Perl extension for the GDAL library for geospatial data

Training and learning



  • Open Geo Tutorial - Tutorial of fundamental remote sensing and GIS methodologies using open source software in python

  • Geoprocessing with Python - GIS circa 2009 - This material is really old and some of it is outdated (not all, though!). One of these days I might get around to putting newer class materials online, but you're stuck with this for now.

Deep learning and Machine Learning

(see Christoph Rieke git hub for much much more)

  • CNN-Sentinel -Analyzing Sentinel-2 satellite data in Python with Keras (repository of our talks at Minds Mastering Machines 2019 and PyCon 2018)

  • Robin Cole on satellite imagery and deep learning resources - Resources for deep learning with satellite & aerial imagery

  • Image patches - Using a U-Net for image segmentation, blending predicted patches smoothly is a must to please the human eye.

  • Fast AI Satellite imagery resources

  • Crop yield prediction - The motive here is to predict the yield of crops of a particular farm by the change in pixels of the image of farm yearly. Uses Tensorflow

  • Houston Flooding with deep learning - Using A Segmentation Neural Net to map out flooded areas of Houston TX using satellite imagery

  • Satellite Imagery Classification with R - Pixel based classification of satellite imagery - feature generation using Orfeo Toolbox, feature selection using Learning Vector Quantization, CLassification using Decision Tree, Neural Networks, Random Forests, KNN and Naive Bayes Classifier

  • SpaceNet building detection - Project to train/test convolutional neural networks to extract buildings from SpaceNet satellite imageries.

  • Road segmentation - Road Detection in satellite imagery. Semantic segmentation is the process of classifying each pixel of an image into distinct classes using deep learning. This aids in identifying regions in an image where certain objects reside.This aim of this project is to identify and segment roads in aerial imagery. Detecting roads can be an important factor in predicting further development of cities, and this concept plays a major role in GeoArchitect (A project which I started). Segmentation of roads is important to map-based applications and is used for finding distances or shortest routes between two places.

  • Super resolution (srcnn) - Super Resolution for Satellite Imagery

  • Pixel decoder - A tool of running deep learning algorithms for semantic segmentation with satellite imagery

  • Detecting ships - Using Satellite Imagery to detect ships (Basic Object Detection)

  • deepOSM - Train a deep learning net with OpenStreetMap features and satellite imagery.

  • Keras for computer vision (Maxime Lenormand GitHub) - Introductions to Keras to perform computer vision tasks, with data exploration, error analysis and improving results.

Great Github accounts with example projects where possible

Please do explore these accounts, there are some absolutely brilliant projects on these accounts, some have more than I have shown here!







  • giswqs - Qiusheng Wu github

  • Python GEE notebooks - A collection of 360+ Jupyter Python notebook examples for using Google Earth Engine with interactive mapping

  • GEE Map - A Python package for interactive mapping with Google Earth Engine, ipyleaflet, and ipywidgets

  • Whitebox Python - WhiteboxTools Python Frontend

  • Lidar - Terrain and hydrological analysis based on LiDAR-derived digital elevation models (DEM)




GDAL of course

  • GDAL Cheat Sheet - Cheat sheet for GDAL/OGR command-line tools

  • GDAL / OGR cookbook - This cookbook has simple code snippets on how to use the Python GDAL/OGR API

  • GDAL tutorial - This blogpost gives in an introduction to GDAL/OGR and explains how the various command line tools can be used.

Earth Observation coding on YouTube

(presenters listed where possible) There are many videos relating to Earth Observation and coding, especially Python. This is really such a small collection of videos here. I have attempted to only include ones with good audio and code examples.

  • xArray at PyConUK2018 - Robin Wilson - Processing thousands of satellite images to understand air quality in the UK - it's efficient and easy with XArray

  • Visualizing & Analyzing Earth Science Data Using PyViz & PyData - Julia Signell - In this talk, we'll work through some specific workflows and explore how various tools - such as Intake, Dask, Xarray, and Datashader - can be used to effectively analyze and visualize these data. Working from within the notebook, we'll iteratively build a product that is interactive, scalable, and deployable.

  • Hands on Satellite Imagery 2019 edition - Sara Safavi - In this tutorial, gain hands-on experience exploring Planet’s publicly-available satellite imagery and using Python tools for geospatial and time-series analysis of medium- and high-resolution imagery data. Using free & open source libraries, learn how to perform foundational imagery analysis techniques and apply these techniques to real satellite data.

  • Python from space - Katherine Scott - In this talk we will work through a jupyter notebook that covers the satellite data ecosystem and the python tools that can be used to sift through and analyze that data. Topics include python tools for using Open Street Maps data, the Geospatial Data Abstraction Library (GDAL), and OpenCV and NumPy for image processing.

  • Remote Sening with Python in Jupyter - In this video we're looking at using Google Earth Engine in Jupyter with the Python API.

  • Writing Image Processing Algorithms with ArcGIS/ArcPy - Jamie Drisdelle - learn how your algorithms can integrate with the raster processing and visualization pipelines in ArcGIS. We’ll demonstrate the concept and discuss the API by diving deep into a few interesting examples with a special focus on multidimensional scientific rasters.

  • Google Earth Engine Python - Qiusheng Wu - Introducing the geemap Python package for interactive mapping with Google Earth Engine and ipyleaflet.

  • Google Earth Engine EE101 Condensed - Noel Gorelick - Introduction to the Earth Engine API and a conceptual overview of key functionality such as compositing, reducing, mapping, zonal statistics and cluminating with building a small app.

  • Image classification with RandomForests using the R languageIn this video I show how to import a Landsat image into R and how to extract pixel data to train and fit a RandomForests model. I also explain how to conduct image classification and how to speed it up through parallel processing.

Earth Engine

EO Geospatial companies or orgs making big contributions

  • Github accounts only with example of work

  • Also contains (Python) libraries for processing satellite data This list aims at highlighting the great work some of the companies / organisations are doing or have done that contribute to the bigger ecosystem. Examples of the accounts work is not exhaustive, some do much more than shown below:

  • development seed

  • Landsat-Util - A utility to search, download and process Landsat 8 satellite imagery

  • GeoLamda - Create and deploy Geospatial AWS Lambda functions


  • mapbox

  • rasterio - Rasterio reads and writes geospatial raster datasets

  • Robosat - Semantic segmentation on aerial and satellite imagery. Extracts features such as: buildings, parking lots, roads, water, clouds








  • PyTroll

  • SatPy - Python package for earth-observing satellite data processing

  • pyresample - Geospatial image resampling in Python



  • CometTS - Comet Time Series Toolset for working with a time-series of remote sensing imagery and user defined polygons

  • SpaceNet6 Baseline - Baseline algorithm for the SpaceNet 6 Challenge



  • Geoscience Australia

  • PyRate - A Python tool for estimating velocity and time-series from Interferometric Synthetic Aperture Radar (InSAR) data.

  • dea-notebooks - Repository for Digital Earth Australia Jupyter Notebooks: tools and workflows for geospatial analysis with Open Data Cube and xarray



QGIS


DEM projects

  • Tin Terrain - A command-line tool for converting heightmaps in GeoTIFF format into tiled optimized meshes.

  • TauDEM - Terrain Analysis Using Digital Elevation Models (TauDEM) software for hydrologic terrain analysis and channel network extraction. Docs

  • DEM.net - Digital Elevation model library in C#. 3D terrain models, line/point Elevations, intervisibility reports. Docs

Radar

  • SAR docker - Source files for Docker image mort/sardocker/

LiDAR

  • pyGEDI - pyGEDI is a Python Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) mission, data extraction, analysis, processing and visualization.

  • GEDI extraction script - Python script to take GEDI level 2 data and convert variables to a geospatial vector format

  • rGEDI - rGEDI: An R Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) Data Visualization and Processing.

  • ICESAT extraction script - Python script to convert from ICESat-2 ATL08 HDF data to shapefile. Usage: 'python icesat2_shp.py

  • ICESAT tools - Tools and code for Icesat-2 data analysis (Python)

Visualisation

Regular blogs of significant interest or posts of interest

EO code Competitions

Useful EO code based twitter accounts

  • pyGEDI - pyGEDI is a Python Package for NASA's Global Ecosystem Dynamics Investigation (GEDI) mission, data extraction, analysis, processing and visualization.

Interesting Non EO parts Python

This bit could potentially become the most valuable resource. Lets not ignore other sectors/industries/data science, instead lets embrace it and learn from all that other amazing stuff!