Checking the MODIS Install

In the previous step we checked out MODIS install. The key here is making sure that .MODIS_Opts.R  has the correct values and is copied the right location. Every time you load MODIS  ( library(MODIS)), the current version of .MODIS_Opt.R will be loaded and those defaults will be used for all your processing.

Next, start your R Studio and load the MODIS library. We are going to check out some  other commands: checkDeps(); .checkTools() and .getDef(). These functions are “hidden from view”  and you don’t usually use them in programming with the package, but since we have the source we can see that they exist, and calling them will give us some more information about the health of our install. To invoke these calls we preceed them with a qualifier MODIS:::   Take a look at the console commands below

> library(MODIS)
> checkDeps()
Error: could not find function “checkDeps”
> MODIS:::checkDeps()
Ok all suggested packages are installed!
> MODIS:::.checkTools()
Checking availabillity of MRT:
‘MRT_HOME’ found: c:/Users/steve/documents/MRT
‘MRT_DATA_DIR’ found: c:/Users/steve/documents/MRT/data
MRT enabled, settings are fine!
Checking availabillity of ‘FWTools/OSGeos4W’ (GDAL with HDF4 support for Windows):
OK, GDAL 1.9.2, released 2012/10/08 found!
> MODIS:::.getDef()

After the last command you should see a listing of all  your MODIS options settings. For now these are controlled by the file .MODIS_Opts.R. If you want to change the organization of your files there is a command to do that “orgStruc()”.  This command allows you to move an entire directory structure to a new location. Before we play with that command, we will need to get some files.

A first attempt at getting files. For this we will use runGDAL().  This command will create the local archive, download hdf files to it and run a gdal process on those file creating  processed output.  for brevity I’ve shortend some of the output. This is taken straight from the sample code.

runGdal( product=”MOD11A1″, extent=”austria”, begin=”2010001″, end=”2010005″, SDSstring=”101″)

No output ‘pixelSize’ specified, input size used!
No ‘resamplingType’ specified, using default: near
No ‘outProj’ specified, using GEOGRAPHIC
No ‘job’ name specified, generated (date/time based)): C:/Users/steve/Documents/MODIS_ARC/PROCESSED/MOD11A1.005_20121125115753
Getting file from: LPDAAC
trying URL ‘ftp://e4ftl01.cr.usgs.gov/MOLT/MOD11A1.005/2010.01.01/MOD11A1.A2010001.h18v04.005.2010007011052.hdf’
using Synchronous WinInet calls
opened URL
downloaded 1.7 Mb

Downloaded by the first try!

The process will continue to download files and has the wonderful feature of retrying downloads that fail. My internet was acting up a bit so one file took 23 tries. I especially like this feature of the package.  When writing  simple downloads from ftp I often find that my calls to  download.file() fail

Getting file from: LPDAAC
trying URL ‘ftp://e4ftl01.cr.usgs.gov/MOLT/MOD11A1.005/2010.01.02/MOD11A1.A2010002.h18v04.005.2010007015858.hdf’
using Synchronous WinInet calls
opened URL
downloaded 4.1 Mb

Downloaded after 23 retries!

As you watch this command execute you will also see output from the “GDAL” process. That will look like this. if you dont see this output, then your path to GDAL is messed up.  ( ha, I messed mine up )  make sure your FWToolspath in MODIS_Opts.R is set to the right location.  I  have both FWTools and OSGeo4W on my system and I set FWTools to the OSGeo4w version of GDal.

Creating output file that is 891P x 307L.
Processing input file HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~1.HDF:MODIS_Grid_Daily_1km_LST:LST_Day_1km.
Using internal nodata values (eg. 0) for image HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~1.HDF:MODIS_Grid_Daily_1km_LST:LST_Day_1km.
0…10…20…30…40…50…60…70…80…90…100 – done.
Processing input file HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~2.HDF:MODIS_Grid_Daily_1km_LST:LST_Day_1km.
Using internal nodata values (eg. 0) for image HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~2.HDF:MODIS_Grid_Daily_1km_LST:LST_Day_1km.
0…10…20…30…40…50…60…70…80…90…100 – done.
Creating output file that is 891P x 307L.
Processing input file HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~1.HDF:MODIS_Grid_Daily_1km_LST:Day_view_time.
Using internal nodata values (eg. 255) for image HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~1.HDF:MODIS_Grid_Daily_1km_LST:Day_view_time.
0…10…20…30…40…50…60…70…80…90…100 – done.
Processing input file HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~2.HDF:MODIS_Grid_Daily_1km_LST:Day_view_time.
Using internal nodata values (eg. 255) for image HDF4_EOS:EOS_GRID:C:\Users\steve\DOCUME~1\MODIS_~1\MODIS\MOD11A1.005\201001~1.05\MOD11A~2.HDF:MODIS_Grid_Daily_1km_LST:Day_view_time.
0…10…20…30…40…50…60…70…80…90…100 – done.

 

Before  I explain the  runGdal() command and all its options, we want to make sure that the process  worked.  Your local modis archive should be easy to find. Mine is in my R Home directory.

Top Level

 

And then when we drill down we will find two directories one for HDF files and a PROCESSED directory for the output from GDAL

 

 

And when we look at the processed files we will see all the files created

 

 

The runGdal() call has thus gone to datapool, selected out the product I wanted ( MOD11A1 or lst ) selected a region of the world which I was able to specify by name “austria”,  selected a time period  day 1 through day 5, and selected out a couple layers of that file. Remember a MODIS HDF file has multiple  Science data layers.  These are specified by SDSstring, more on that later

 

  1. December 11, 2012 at 4:43 PM

    Hi Steve,

    Many thanks for this tutorial. It is a real help for installing the MODIS package. I am going to follow your blog closely for more useful posts. If I have anything to contribute in the way of example scripts I am happy to share them with you in the future.

  2. Philipp
    April 29, 2013 at 8:53 PM

    Hi Steve,

    great tutorial, thanks a lot. I followed your steps but run into a problem with the MODIS_Opts.R gdalpath.

    I use MODIS 0.8-07 and I found two MODIS_Opts.R files (both have gdalPath=”C:/OSGeo4W/bin”)
    1. in ~\Documents\R\win-library\3.0\MODIS\external
    2. in ~\Dokuments

    I also set gdalPath environmental variable (C:\OSGeo4W\bin)

    with: MODIS:::checkTools()
    Checking availabillity of ‘FWTools/OSGeo4W’ (GDAL with HDF4 support for Windows):
    OK, GDAL 1.7.3, released 2010/11/10 found!

    with: MODISoptions(), I get
    PROCESSING
    GDAL : disabled. Use ‘MODIS:::checkTools(‘GDAL’)’ for more information!

    when I apply runGdal(etc.), I get

    Error in MODIS:::checkIntegrity(x = x[a], …) :
    Your GDAL installation or the path to your GDAL/bin directory is not valid, please set it using ‘?MODISoptions’

    Any idea where it is going wrong?

    Regards

    • Steven Mosher
      May 9, 2013 at 1:03 AM

      i’m not sure, I havent been keeping up with the package development for sometime. you might contact the maintainer or have a look at the source

  3. Andreas Westergaard
    June 27, 2013 at 2:15 PM

    Hi. Really a thorough guide, thanks.
    Now that USGS has shut down the possibility for ftp download, do you plan to update the guide to acquire MODIS from http? Thanks
    -Andreas

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 36 other followers

%d bloggers like this: