Re: call_external for file I/O

[David Foster <foster(at)bial1.ucsd.edu>]

Charlie Solomon wrote:
> The data is in a disk file initially, the map_file routine puts the relevant
> part of the file into memory and sets those pages for sequential access, so
> that when IDL accesses the data in memory it is not based on an algorithm
> like most recently used.  This is optimal for image data in which you may
> need to access different subsets of the array at different times.
> 
> I agree that it is fast, but the method we use on our Solaris machine seems
> much faster for the same files.  I don't have time data available right now,
> but I could get it if needed.

Charlie -

I'm wondering if what you are coming up against is a storage order
problem, in which the elements of the array are being accessed in
an order that is not optimal given the order of storage of the
elements in memory. Ideally the dimension that varies fastest in
memory should be incremented first.

We've run up against this problem before, and have written a C
routine to optimize access to a 3D array in memory. If you're
interested in checking it out let me know.

You also might want to look into Eric Korpela's VARRAY utility
that allows you to have memory mapped files:

	http://sag-www.ssl.berkeley.edu/~korpela/mmap/
	korpela@islay.ssl.berkeley.edu

Dave
-- 

  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    David S. Foster         Univ. of California, San Diego
     Programmer/Analyst     Brain Image Analysis Laboratory
     foster@bial1.ucsd.edu  Department of Psychiatry
     (619) 622-5892         8950 Via La Jolla Drive, Suite 2240
                            La Jolla, CA  92037
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~