文件列表:

preprocessing/
tests/
run_tests_gadi.sh

# geo-wavelets 2D Wavelet decomposition/reconstruction for raster data ## Running on Gadi: Load modules and define paths (amend as required) as follows for interactive runs: ``` module purge module load python3/3.7.4 module load gdal/3.0.2 module load openmpi/4.1.4 module load hdf5/1.10.5p source /g/data/ge3/ml_preproc_venv/bin/activate export GDAL_DATA=/apps/gdal/3.0.2/share/gdal/ export PYTHONPATH=/apps/gdal/3.0.2/lib64:/apps/gdal/3.0.2/lib64/python3.7/site-packages export LC_ALL=en_AU.UTF-8 export LANG=en_AU.UTF-8 export PYTHONPATH=/g/data/ge3/rakib/ML-preprocessing:$PYTHONPATH ``` Amend the last line above to point to a local copy of ML-preprocessing. For `pbs` batch jobs, check example pbs scripts in the `preprocessing/pbs` folder in this repository. ## Running Tests on Gadi: Note that if you have a local version of Python3 installed in your environment, running `pytest` in the `tests` folder will likely fail, with `gdal` being reported as missing. In such cases, run `pytest` as follows: `python3 -m pytest` The above will run the system version of `pytest` and the tests should all run to completion. ## User Parameters `multiscale.py` takes the following input parameters: ``` Usage: multiscale.py [OPTIONS] INPUT OUTPUT_FOLDER MAX_LEVEL INPUT: Path to raster files, or a file containing a list of raster file names (with full path) OUTPUT_FOLDER: Output folder MAX_LEVEL: Maximum level up to which wavelet reconstructions are to be computed. Each consecutive level halves the raster resolution (doubles the spatial wavelength); thus the spatial wavelength at a given level is given by: spatial_wavelength = orig_spatial_wavelength * 2^level Example usage: mpirun -np 2 python multiscale.py filelist.txt /tmp/output 10 --max-search-dist 500 Running in Cluster Environments: This script requires all raster data to be loaded into memory for processing. Furthermore, each raster being processed requires 4 times as much memory, e.g. a 7 GB raster will require ~28 GB of RAM. Hence, for parallel runs, one must be judicious in terms of allocating processors, bearing in mind the amount of memory available on a given compute node on the cluster. Options: --file-extension TEXT File extension to use (e.g. '.tif') to search for input files; only applicableif the 'input' argument is a folder. --mother-wavelet [bior1.1|bior1.3|bior1.5|bior2.2|bior2.4|bior2.6|bior2.8|bior3.1|bior3.3|bior3.5|bior3.7|bior3.9|bior4.4|bior5.5|bior6.8|coif1|coif2|coif3|coif4|coif5|coif6|coif7|coif8|coif9|coif10|coif11|coif12|coif13|coif14|coif15|coif16|coif17|db1|db2|db3|db4|db5|db6|db7|db8|db9|db10|db11|db12|db13|db14|db15|db16|db17|db18|db19|db20|db21|db22|db23|db24|db25|db26|db27|db28|db29|db30|db31|db32|db33|db34|db35|db36|db37|db38|dmey|haar|rbio1.1|rbio1.3|rbio1.5|rbio2.2|rbio2.4|rbio2.6|rbio2.8|rbio3.1|rbio3.3|rbio3.5|rbio3.7|rbio3.9|rbio4.4|rbio5.5|rbio6.8|sym2|sym3|sym4|sym5|sym6|sym7|sym8|sym9|sym10|sym11|sym12|sym13|sym14|sym15|sym16|sym17|sym18|sym19|sym20] Name of the mother wavelet --extension-mode [zero|constant|symmetric|reflect|periodic|smooth|periodization] Signal extension mode used for padding --extrapolate BOOLEAN Extrapolate raster if NO_DATA_VALUES are found. 'Ringing' artefacts can result near sharp contrasts in image values -- especially at the edges of NO_DATA_VALUE regions. By extrapolating image values to regions of NO_DATA_VALUE, 'ringing' artefacts can be pushed further outward, away from the region of interest in the original image. Raster data are extrapolated by default and this parameter has no effect when the input raster has no masked regions --max-search-dist INTEGER Maximum search distance (in pixels) for extrapolating image values to regions of NO_DATA_VALUE in input raster; not used if raster has no masked regions --smoothing-iterations INTEGER Number of smoothing iterations used for smoothing extrapolated values; see option --max-search-dist --keep-level INTEGER Level to keep. Note that by default all levels up to max-level are saved, which may cause disk space issues. This option allows users to save only those levels that are of interest; e.g. to keep only levels 5 and 6, this option must be repeated twice for the corresponding levels --log-level [DEBUG|INFO|WARN] Logging verbosity -h, --help Show this message and exit. ```

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：