Reading a file to its end

Here I give a recipe to solve a common problem: how to read a file to its end when you don't know how many lines it contains and you don't accept an IDL crash (whish is, I agree a fair expectation).

I must admit I did not find that myself and I had to dig in IDL's libraries to find an example. I found it in the routine xdisplayfile.pro.

The idea is to track the input/output status of the readf command and stop the process when an error is detected. The lines are stored in an array which is trimmed at the end. So here is the code:

	openr,unit,the_file,/get_lun

	the_lines = strarr[1000]

	on_ioerror,done_reading
	readf,unit,the_lines

	done_reading: s = fstat(unit)
	the_lines = the_lines[0:(s.transfer_count-1)>0]
	on_ioerror,null

	free_lun,unit

The incovenient of this method is that you need to first declare an array that will contain all the lines (i.e. an oversized array). However, unless there is a system call available that will tell you the number of lines (such as wc -l on UNIX) all other tricsk boil down to reading the file twice.

There is an alternate, less cryptic method, that I have also stolen this time in the ASTROLIB, that uses the eof function. I admit that it is more elegant so here it is:

	openr,unit,file,/get_lun
	
	while not eof(unit) do begin
		readf,unit,this_line
		if (n_elements(the_lines) eq 0) then begin
			the_lines = this_line
		endif else begin
			the_lines = [the_lines, this_line]
		endelse
	endwhile
	free_lun, unit

In this example, I am just appending the new line to an array that I am creating on-the-fly, but since you are reading the file line per line, you may as well process the information, or read detailed information (numbers, strings, ...). The fact that you are testing for whether the end-of-file has been reached before reading a new element ensures that no garbage is added to your input.