The -LINES= n option (or -LN= n) paginates the listing for printing. Use this to change the number of lines per page. Specifying -LINES=0 paginates at subroutine boundaries.

If you do not specify the -LINES option, PFA prints 55 lines per page.

The -LISTOPTIONS= list option (or -LO= list) specifies the information to include in the listing file (.l), where list is any combination of the options in Table 3-1.

Use the -SUPPRESS= list option (or -su= list) to disable individual classes of PFA messages that are normally included in the listing (.l) file. These messages range from syntax warnings and error messages to messages about the optimizations performed. list is any combination of the options in Table 3-2.

If you do not specify this option, PFA prints messages of all classes.

The listing file is in 132-column format. To view the file, open a window with 132 columns and 40 rows by entering

% wsh -s132,40

This section explains the contents of the .l file when you use the default values for the -LISTOPTIONS command line option (that is, O and L).

A default PFA file listing includes

DO Loops  Line
+-------  173      DO 100 M=2,MAX(MFLD,2)
|         174      IADR = ISECT(M)
|         175      IADR1= ISECT(M-1)
|         176      PNM(IADR)=(ANM(IADR) *PNM(IADR1))
|_______  177 100  PPNM(IADR)= -(ANM(IADR) *PNM(IADR1))

13 DD   1790  IF (B(I) .LE. 6) IB(J*I) = I+J

13: data dependence    Data dependence involving this line due
                       to variable IB.

Footnotes Actions   DO Loops   Line
                                 1      SUBROUTINE Z(A,B,N)
                                 2      REAL A(N), B(N)
                    +-------     3      DO 20 I=1,N
                    !            4      X=A(I)
                    !            5      Y=B(I)
                    ! ______     6   20 C(I)=X+Y
### line (6) 
### error    Array not declared or statement function declared 
             after executable statements.
### error    A do loop ends on a non-executable statement.
                                 7      PRINT *,X
                                 8      END

PFA cannot determine if it can run a loop in parallel when the code uses indirect indexing. A loop is indirectly indexed when it uses the value from some auxiliary array as the index value rather than the DO loop variable.

The Fortran 77 code

subroutine foo2(w,b,index,n)
real w(n), b(n)
integer index(n)
do i = 1, n
w(index(i)) = w(index(i)) + b(i)
enddo
end

when submitted to PFA, results in the listing file

                      10
                      11
                      12    subroutine foo2(w,b,index,n)
                      13    real w(n), b(n)
                      14    integer index(n)
                      15
1 Q       +-------    16    do i = 1, n
2 DD      !           17       w(index(i)) = w(index(i)) + b(i)
          !_______    18    enddo
                      19    end
Abbreviations Used
DD   data dependence
Q   question
Footnote List
1: question           Is INDEX a permutation vector?
2: data dependence    Data dependence involving this line due
                      to variable W.
DO Loop Summary
loop# from   to   DO label index   workload status
1     16     18   DO       I       dependencies prevent
                                   parallelism

DD in the Actions column on line 17 of the listing warns that the variable w might carry a dependency. A dependency exists when one iteration of the loop writes to a location that is used by a different iteration of the loop. In this example, if the values of index(i) are ever the same for different values of i, then different iterations might use the same location in w. Therefore, this code contains a possible data dependence.

If you can guarantee that the values of index(i) are always different for each value of i, then there is no dependence (each iteration uses a different location in w). Question one on the Footnote List asks if index(i) is different for every value of i. A permutation vector is a list of numbers, each of which is different from the others. If you know that index is a permutation vector, then the loop is data-independent. An example of a permutation vector is a list of objects in which each object appears exactly once.

Explicitly state that index is a permutation vector by adding an assertion in the source

subroutine foo2(a,b,index,n)
real a(n), b(n)
integer index(n)
c*$*assert permutation (index)
do i = 1, n
a(index(i)) = a(index(i)) + b(i)
enddo
end

Now the listing file shows that PFA finds the loop safe to run in parallel (indicated by the * DO loop delimiter)

Actions  DO Loops   Line
DIR                  1  #  1 “foo2.f”
                     2     subroutine foo2(a,b,index,n)
                     3     real a(n), b(n)
                     4     integer index(n)
                     5
DIR                  6 c*$*assert permutation (index)
C        +------     7     do i= 1, n
         *           8        a(index(i)) =  a(index(i)) + b(i)
         *______     9     enddo
                     10    end

Abbreviations Used

DIR   directive
C     concurrentized

Loop Summary

        From   To     Loop    Loop
Loop#   line   line   label   index   Status
1       7      9      Do      I       concurrentized

---

Note: As with all assertions, PFA does not verify the truth of this assertion. When you make an assertion, be certain that the assertion is always true for all possible input data.

---

This example shows what happens when a loop contains a call to an external routine. The Fortran 77 code

subroutine foo3 (a,b,c,n)
real a(n), b(n), c(n)
external force
do i = 1, n
a(i) = force (b(i), c(i))
enddo
end

generates the listing

Actions DO Loops     Line
DIR                  1 #  1 “foo3.f”
                     2    subroutine foo3(a,b,c,n)
                     3    real a(n), b(n), c(n)
                     4    external force
                     5
NCS     +------      6    do i = 1, n
NO NCS  !            7       a(i) = force(b(i), c(i))
        !______      8    enddo
                     9    end
Abbreviations Used
NO   not optimized
DIR   directive
NCS   non-concurrent-stmt
Footnote List
1: not optimized       No optimizable statements found.
2: not optimized       Unoptimizable call to “FORCE” found.
Loop Summary
        From   To     Loop    Loop
Loop#   line   line   label   index   Status
1       6      8      Do      I       unoptimizable
                                      call (FORCE)

Calling the function force prevents PFA from automatically running the loop in parallel. PFA identifies the function call as a non-concurrent-stmt. By its nature, a nonconcurrent statement prevents PFA from assuming the loop is safe to run in parallel because PFA cannot see into the routine to look for data dependencies.

If you know that force generates no data dependencies, then explicitly state this fact for the nonconcurrent statement

subroutine foo3(a,b,c,n)
real a(n), b(n), c(n)
external force
c*$*assert concurrent call
do i = 1, n
a(i) = force(b(i), c(i))
enddo
end

Now that PFA knows that the nonconcurrent statement involves no data dependency, PFA will find the loop safe to run in parallel.

There is one subtlety in using the concurrent call assertion. When you use this assertion, PFA makes no attempt to examine the called routine; it simply assumes that it is safe. However, PFA is still left with the problem of correctly declaring the variables in the loop to be either SHARE or LOCAL. (PFA does the best it can, but it can sometimes be fooled.) For example,

subroutine tricky (a,b,c,n,m)
real a(*), b(*)
external my_function
c*$*assert concurrent call
do i = 1, n
a(i) = my_function (b(i), m)
b(i) = a(i) + m
enddo
m = 0
end

The question is whether the variable m should be SHARE or LOCAL. If the routine my_function only reads the old value of m, then it should be SHARE. If my_function writes a new value of m, then it should be LOCAL. In the absence of any more clues, PFA must go by what it can see; and what it can see is that within the loop, there are no visible assignments to m, and so PFA will declare it to be SHARE. If in fact my_function is writing the value of m, then this is incorrect. In this case, to give PFA the hint it needs, add a visible assignment to m at the top of the loop.

For example, consider the following code:

do i = 1, n
   m = 0
   a(i) = my_function(b(i), m)
b(i) = a(i) + m
enddo

Here, PFA can see an assignment to m and so will declare it to be LOCAL. Note that if my_function is both reading the old value and writing a new value of m, then it was not legal to parallelize the loop.

This example shows how PFA produces a single value from a set of values. Because the entire set of values is reduced to a single value, these operations are called reductions.

Consider the Fortran 77 code

subroutine foo4(a,b,n,sum)
real a(n), b(n), sum
sum = 0.0
do i = 1, n
sum = sum + a(i)*b(i)
enddo
end

Using the previous code as input, PFA produces the listing file

DIR             1 # 1 “foo4.f”
                2      subroutine foo4(a,b,n,sum)
                3      real a (n), b(n), sum
                4
                5      sum = 0.0
      +-----    6      do i = i, n
1 DD  !         7         sum = sum + a(i)*b(i)
      !_____    8      enddo
                9      end
Abbreviations Used
DD        data dependence
DIR       directive
Footnote List
1: data dependence       Data dependence involving this
                         line due to variable “SUM”.
Loop Summary
       From  To    Loop   Loop
Loop#  line  line  label  index   Status
1      6     8     Do     I       scalar mode preferable

Because different iterations of the loop read and write the same location (the variable sum), there is a dependence. However, this is a special case. Because sum just accumulates a total, you can accumulate subtotals in parallel and then combine the subtotals at the end.

Because the parallel version of the code adds the elements together in a different order than the single-process version, the round-off errors accumulate differently for the two versions of the code. Thus, the answer can differ slightly as you vary the number of processes used to run the code. In fact, if you use the dynamic scheduling option for the code, the answer might vary slightly from one run of the program to the next, even if you use the same number of processes on the same machine.

Most applications can safely ignore this variation in round-off error. If you do not care about this round-off error, you can tell PFA to use parallel subtotals. To tell PFA not to worry about round-off error, you can use either the C*$*ROUNDOFF=2 directive or the f77/pfa command line option -WK , -roundoff=2.

The resulting listing file is

DIR              1 # 1 “foo4.f”
                 2      subroutine foo4(a,b,n,sum)
                 3      real a(n), b(n), sum
                 4
                 5      sum = 0.0
C      +------   6      do i = 1, n
       *         7         sum = sum + a(i)*b(i)
       *______   8      enddo
                 9      end
Abbreviations Used
DIR       directive
C         concurrentized
Loop Summary
       From  To    Loop   Loop
Loop#  line  line  label  index  Status
1      6     8     Do     I      concurrentized

Be aware that the round-off error produced by the parallel reduction operation is not necessarily any worse than the round-off error already present in the original serial version. It will simply be different. If your application did not worry about the round-off error in the original, there is no reason to suppose that it should worry about it in the parallel version. If, on the other hand, your application takes special steps to reduce round off (for example, adding the numbers together in order from smallest absolute value to largest), then you should not use parallel reductions.

The previous example is called a sum reduction because the reduction operator is +. Table 3-5 shows the types of reductions PFA supports.

All these reductions are under the control of the -ROUNDOFF command line option, even though technically the min and max reductions do not involve round-off problems.