This chapter has the following major topics:
“Selecting Files to Generate” tells how to select output files.
“Using the Transformed Source File” describes the source file generated by Power Fortran.
“Using the Workshop Pro MPF File” covers that file.
“Controlling the Power Fortran Listing File” discusses listing options.
“Interpreting the Listing” discusses the listing contents.
“Sample Listings” illustrates the use of the listing file in solving some Power Fortran problems.
Power Fortran generates the types of output files shown in Table 2-1.
File Contents | Suffix | Driver Option Required |
|---|---|---|
Listing of program structure and message | .L | -pfalist or -pfakeep |
Transformed Fortran source file | .m | -pfakeep |
Input file for WorkShop Pro MPF | .anl | -pfakeep |
Suppose that the file sample.f90 contains the tiny subroutine in Example 2-1.
subroutine sample (a,b,c)
dimension a(1000),b(1000),c(1000)
do i = 1, 1000
a(i) = b(i) + c(i)
end do
end
|
When sample.f is compiled using the following command:
f90 -64 -pfakeep -c sample.f |
the output includes the following files:
sample.L contains the listing file
sample.m contains the transformed source
sample.anl contains the WorkShop Pro input file
The transformed source file contains the original source lines plus parallel-execution directives generated by Power Fortran. The sample.m file produced from Example 2-1 contains the lines shown in Example 2-2.
C KAP/SGI_F90 MP 00.00 k240000 190195o5r0so3 20-Jun-1995 14:54:32
# 1 "pfa1.f90"
subroutine SAMPLE ( A, B, C )
integer I
real A(1000), B(1000), C(1000)
C$PAR parallel shared (A,B,C) local (I)
CSGI$ startloop 3
C$PAR pdo
do 2 I=1,1000
A(I) = B(I) + C(I)
2 continue
C$PAR end pdo nowait
CSGI$ endloop 3
C$PAR end parallel
end
|
The transformed file contains numerous generated directives. These directives, such as C$PAR, direct the parallel execution of the compiled program. Many of these directives are documented in the MIPSpro Fortran 90 Programmer's Guide (for example, C$PAR is a PCF directive that marks the start and end of a parallel work unit). However, some directives are not intended for general use and are not documented (for example, C$SGI is not documented).
You use the transformed source file for information only. The transformed source uses fixed format and loops in the Fortran 77 style, instead of the Fortran 90 syntax. As a result, it might not compile correctly if your were to change its suffix to .f90 and compile it.
The WorkShop Pro MPF Parallel Analyzer View cvpav helps you understand the structure and operation of parallelized applications by providing an interactive, visual comparison of their original source with transformed, parallelized code. The Parallel Analyzer View reads the .anl analysis files generated by Power Fortran and displays editable parameters for each DO loop. These parameters are easily customized and explored with the help of the Parallel Analyzer View's user-friendly graphical interface.
A sample of a .anl file is shown in Example 2-3.
product "KAP/SGI_F90 MP " version "00.00 k240000 190195" routine "SAMPLE" line 2 - 7 begin_orig_loop_section "SAMPLE" orig_loop 1 "SAMPLE" do nest 1 range 4 - 6 index "I" loop_option 1 optimize 5 loop_option 1 roundoff 0 loop_option 1 scalar_optimize 3 loop_option 1 limit 20000 loop_option 1 arc_limit 5000 loop_option 1 minconcurrent 1000 loop_option 1 unroll 4 loop_option 1 unroll_weight 100 loop_option 1 max_invariant_if_growth 500 loop_option 1 each_invariant_if_growth 20 variable_list 1 "C":r , "B":r , "A": w, "I":rw end_orig_loop 1 end_orig_loop_section "SAMPLE" begin_transformed_loop_section "SAMPLE" transformed_loop 3 1 "SAMPLE" ordinary workload 5 schedtype simple "Default" mode parallel nest 1 line_range 4 - 6 end_transformed_loop 3 "SAMPLE" end_transformed_loop_section "SAMPLE" begin_vln file_name 1 "pfa1.f90" vln_translation 1 1 1 vln_translation 1 2 1 end_vln |
The Power Fortran listing file shows the input source statements and how Power Fortran treated them. It is your primary tool for judging the effect of parallelization, and for discovering how to make Power Fortran more effective. The listing file produced by Example 2-1 is shown in Example 2-4.
Footnotes Actions DO Loops Line
DIR 1 # 1 "pfa1.f90"
1 subroutine sample (a,b,c)
2 dimension a(1000),b(1000),c(1000
C +--------- 3 do i = 1, 1000
* 4 a(i) = b(i) + c(i)
*_________ 5 end do
6 end
Abbreviations Used
DIR directive
C concurrentized
Loop Summary
From To Loop Loop at
Loop# line line label index nest Status
1 3 5 Do I 1 concurrentized
|
The page-heading lines and some blank lines have been deleted from Example 2-4. The output in Example 2-4 is the result of using default listing options. A listing with more details is shown under “Interpreting the Listing”.
 | Note: The first line 1, containing a number directive, was actually added by cpp. The source file passes through cpp prior to Power Fortran. |
The driver option –lines=n (or –ln=n) paginates the listing at n lines per page. When you specify –lines=0, the listing is paginated at subroutine boundaries.
If you do not specify the –lines option, Power Fortran writes 55 lines per page.
The driver option –listoptions=list option (or –lo=list) specifies the information to include in the listing file. The list is any combination of the option letters in Table 2-2. The default is –listoptions=ol.
Table 2-2. Listing File Include Options
Letter | Controls This Information |
|---|---|
c | Calling tree at the end of the program listing. |
i | Transformed program file annotated with line numbers in the source program. Error messages and debugging information can refer to the original source rather than the transformed source. This argument is specified by default. |
k | List of the Power Fortran options used at the end of each program unit. |
l | Loop-by-loop optimization table. |
n | Program unit names, as processed, to the standard error file. This option is added automatically as part of an f90 –v compilation. |
o | Annotated listing of the original program. |
p | Processing performance statistics. |
s | Summary of optimizations performed. |
t | Annotated listing of the transformed program. |
The following command compiles the program source.f90 with Power Fortran and includes an annotated listing of the original program and a summary of the optimizations performed in the listing file:
% f90 -pfa -lo=ls source.f90 |
Use the –suppress=list option (or –su=list) to disable individual classes of Power Fortran messages that are normally included in the listing. These messages range from syntax warnings and error messages to messages about the optimizations performed. list is any combination of the option letters summarized in Table 2-3.
Table 2-3. Listing File Message Disabling Options
Value | Message Class Disabled |
|---|---|
d | Data dependence |
e | Syntax error |
i | Information |
n | Unable to run loop in parallel |
q | Questions |
s | Standard messages |
w | Warning of syntax error (Power Fortran adds the –suppress=w option automatically if you use the –w option to f77) |
If you do not specify this option, Power Fortran prints messages of all classes.
The listing file is in 132-column format. To view the file, open a window with 132 columns and the number of rows in a page by entering an xwsh command with specific geometry.
% xwsh -geometry 132x55 |
Optionally you can include a font specification in order to reduce the size of the window while retaining readability.
% xwsh -fn "-*-screen-medium-r-normal--12" -geometry 132x40 |
When you want to display the listing file from a shell script or makefile, you can consider other options described in the xwsh(1) reference page. The command fragments in Example 2-5 might appear in a shell script. This xwsh starts up quickly because it does not open a shell. The user browses the listing using the xwsh scroll buffer, whose size is set to 5000 lines in order to accommodate large listings.
f90 ... -pfalist -lines=40 ... -c $FILENAME.f90
xwsh -fn "-*-screen-medium-r-normal--12" \
-geometry 132x40 \
-sl 5000 \
-title "$FILENAME listing" \
-nokeyboard -nomenu -hold \
-e cat $FILENAME.L
|
The listing file generated by Power Fortran lists the changes Power Fortran made to the code, and shows where and why the code was not changed. For example, a message could say that, although three loops could have run in parallel, Power Fortran converted only the one it determined most profitable. When you understand the information in the Power Fortran listing, you can recognize where small changes to the source code will make a difference in which code is run in parallel.
Any Power Fortran listing that includes the includes the default list options o and l (as described under “Specifying Optional Information”) contains the following fields:
source lines with line numbers
DO loop annotations
action summary
footnotes
syntax errors/warning messages
The listing fragment in Example 2-6 illustrates these fields.
Footnotes Actions DO Loops Line
16
17 integer,parameter::adim = 1000
18 real ra(adim),rb(adim),rc(adim)
19
20 ! load the arrays -- suboptimal i/o for example
21 open (unit=11,action="READ",file="pfa1.data")
1 NO NCS +--------- 22 do i = 1,adim
2 NO NCS ! 23 read (unit=11,fmt="2E15.8") ra(i),rb(i)
!_________ 24 end do
25
26 ! swizzle the contents
27 ! function has no side-effects
DIR 28 !*$* ASSERT CONCURRENT CALL
C +--------- 29 do i = 1,adim
* 30 rc(i) = swizzle(ra(i),rb(i))
*_________ 31 end do
32
33 ! un-swizzle in subroutine
34 call subber(ra,rb,rc)
35
36 end
Abbreviations Used
NO not optimized
DIR directive
NCS non-concurrent-stmt
C concurrentized
Footnote List
1: not optimized This loop contains unoptimizable input or output statements.
2: not optimized This input or output statement inhibits loop optimization.
|
A statement in the listing with a line number, such as 21, is either that line from the original source file, or else it is derived from that line. Line numbers are useful when inspecting the transformed program listing and when debugging.
Power Fortran sometimes generates several lines of code from a single line of the original program. In this case, each generated line is labeled with the same number as the line of the original source from which it was generated.
The listing displays the boundaries of DO loops graphically in the column headed “DO Loops.” Power Fortran surrounds each loop (up to a nesting level of 10) with a loop delimiter character. The delimiters form brackets around each loop nest level. The delimiter characters are listed in Table 2-4.
Table 2-4. Listing File DO Loop Delimiters
Character | Denotes |
|---|---|
| | Generic DO loop |
* | Power Fortran can run loop in parallel |
! | Problem preventing optimization |
A statement contained within n DO loops has n of these loop delimiters on that line. You can use the delimiters to trace the extent of each loop, and to quickly learn which loops were parallelized.
When Power Fortran translates or modifies a statement, it places abbreviations in the “Actions” column of the listing to explain what was done to that statement. The abbreviations used in this column are expanded at the bottom of each page. Table 2-5 lists and explains the values that can appear in the Actions column.
Table 2-5. (Continued) Power Fortran Action Abbreviations
Value | Meaning |
|---|---|
C | Concurrentized: This statement will execute on multiple CPUs. |
DD | Data Dependence: Data dependence prevents Power Fortran from running this statement in parallel. |
DEL | Deleted: ? |
DIR | Directive: This statement recognized as a directive. If you code a directive and that line does not have the DIR abbreviation, Power Fortran did not recognize the directive. Check the setting of the –directives driver option and the syntax of the directive. |
E | Error: Error found in this statement. The message can refer to missing or extra characters, illegal keywords, or text placed in the wrong column. The transformed source file (.m) contains an unmodified copy of this line. |
EX | Extension: A construct in the original program is a Fortran language extension not supported in the language that Power Fortran generates.. |
I | Inserted statement. |
INF | Information message. See the related listing footnote.. |
INL | Inlined: Statement inserted in the inlining process. |
IPA | Interprocedural analysis: Statement inserted or modified during inlining or IPA. |
LM | Label modification: Loop target label generated or modified. |
LR | Loop Reordering: This DO statement has been moved in an nterchange with a nested or containing loop. |
MIS | Miscellaneous: Unspecific message. This message does not always mean that something is wrong with the program. |
NC | Not Concurrentized: this statement could not be made concurrent for the reason shown in another code. |
NO | Not Optimized: some Power Fortran internal data structure or table has overflowed. To process this source, you must split the file into smaller sections. |
NT | Not Tiled: This statement could not be optimized for cache performance. |
NV | Not Vectorized: This statement could not be optimized for vector performance. |
OE | Option Error: An error detected in Power Fortran driver option. This error does not stop processing of a program unit. |
OPT | Optimized: This statement modified by scalar optimization. |
OT | Output Translation Failure: This statement cannot be represented in the Fortran language generated by Power Fortran. |
OW | Option Warning: A questionable condition detected in a Power Fortran driver option. |
Q | Question: This statement cannot be optimized unless more information is given. The question is given in a footnote line. You can usually answer this question with an appropriate assertion. |
SC | Scalar Optimization: This statement modified by scalar optimization.. |
STD | Standardized: This statement was transformed into a standard Fortran form in order to improve the changes of being able to optimize it. |
T | Tiled: This statement transformed to improve cache memory performance. |
TE | Translator Error: Indicates an internal Power Fortran error. Power Fortran writes the notification to the standard error file and writes a trace back to the output file. Notify Silicon Graphics if you see this message. |
TL | Too Large: The source program is too large; Power Fortran can only process it in smaller sections. |
V | Vectorized: This statement modified for vector performance. |
W | (Warning) Contains syntax warnings. |
The “Footnotes” column gives details concerning the actions or nonactions taken. Power Fortran numbers and prints the footnotes at the bottom of each program unit under the Footnote List heading. For example, the following line from a listing contains the footnote number 13.
13 DD 1790 IF (B(I) .LE. 6) IB(J*I) = I+J |
In the Footnote List displayed at the end of the program unit, the footnote appears.
13: data dependence Data dependence involving this line due
to variable IB.
|
In this example, 13 is the footnote number, DD (data dependence) is the explanation for the action (in this case, nonaction).
When a program has errors, the listing contains the error text following the line in error. The same messages are printed to stderr during the compile. The following example displays a listing with an error.
Footnotes Actions DO Loops Line
1 SUBROUTINE Z(A,B,N)
2 REAL A(N), B(N)
+------- 3 DO 20 I=1,N
! 4 X=A(I)
! 5 Y=B(I)
! ______ 6 20 C(I)=X+Y
### line (6)
### error Array not declared or statement function declared
after executable statements.
### error A do loop ends on a non-executable statement.
7 PRINT *,X
8 END
|
This section contains a few simple examples of Fortran code and the corresponding Power Fortran output. An actual source program would be much larger, and a single loop could contain several of the cases illustrated here. However, even in a large loop, you can deal with each problem individually.
Power Fortran cannot determine if it can run a loop in parallel when the code uses indirect indexing. A loop is indirectly indexed when it uses the value from some auxiliary array as the index value rather than the DO loop variable. The code in Example 2-7 illustrates indirect indexing.
subroutine indirex(a,b,index,n)
real a(n), b(n)
integer index(n)
do i = 1, n
a(index(i)) = a(index(i)) + b(i)
end do
end
|
When Example 2-7 is submitted to Power Fortran, it produces the listing shown in Example 2-8.
Footnotes Actions DO Loops Line
DIR 1 # 1 "pfa3.f90"
1 subroutine INDIREX ( A, B, INDEX, N )
I 1 integer N, I
I 1 real A(N), B(N)
I 1 integer INDEX(N)
I 4 CSGI$ startloop 2
1 Q +--------- 4 do I=1,N
2 DD ! 5 A(INDEX(I)) = A(INDEX(I)) + B(I)
!_________ 6 end do
I 6 CSGI$ endloop 2
7 end
Abbreviations Used
DD data dependence
Q question
I inserted
DIR directive
Footnote List
1: question Is "INDEX" a permutation vector?
2: data dependence Data dependence involving this line due to variable "A".
|
DD in the Actions column for statement 6 warns that the variable A might carry a dependency. A dependency exists when one iteration of the loop writes to a location that is used by a different iteration of the loop. In this example, if the values in index are not unique, different iterations might write to the same element of A.
The question given in footnote 1 asks if index is a permutation of the integers 1..n. (Since all elements index(i) are used as subscripts of a, Power Fortran assumes they are all taken from the set 1..n; however only a permutation of 1..n contains no duplicate values.)
If you are sure that index is a permutation vector, there can be no data dependence between elements of A. You inform Power Fortran of this by adding an assertion, as shown in Example 2-9.
subroutine indirex(a,b,index,n)
real a(n), b(n)
integer index(n)
!*$* assert permutation (index)
do i = 1, n
a(index(i)) = a(index(i)) + b(i)
end do
end
|
The listing in Example 2-10 shows that Power Fortran finds the loop safe to run in parallel.
Footnotes Actions DO Loops Line
DIR 1 # 1 "pfa3.f90"
1 subroutine INDIREX ( A, B, INDEX, N )
I 1 integer N, I
I 1 real A(N), B(N)
I 1 integer INDEX(N)
DIR 4 C*$* assertpermutation ( INDEX )
I 5 C$PAR parallel if (N .gt. 142) shared (N,INDEX,A,B) local (I)
I 5 CSGI$ startloop 3
I 5 C$PAR pdo
LM I C +--------- 5 do 2 I=1,N
* 6 A(INDEX(I)) = A(INDEX(I)) + B(I)
I *_________ 7 2 continue
I 7 C$PAR end pdo nowait
I 7 CSGI$ endloop 3
I 7 C$PAR end parallel
8 end
Abbreviations Used
LM label modification
I inserted
DIR directive
C concurrentized
|
| Note: Power Fortran cannot verify the truth of this, or any, assertion. When you make an assertion, it is up to you to be certain that it is true for all possible input data. |
The code in Example 2-11 contains a call to an external function.
subroutine caller(a,b,c,n)
real a(n), b(n), c(n)
external force
do i = 1,n
a(i) = force(b(i),c(i))
end do
end
|
The Power Fortran listing for this code is shown in Example 2-12.
Footnotes Actions DO Loops Line
DIR 1 # 1 "pfa4.f90"
1 subroutine caller(a,b,c,n)
2 real a(n), b(n), c(n)
3 external force
1 NO NCS +--------- 4 do i = 1,n
2 NO NCS ! 5 a(i) = force(b(i),c(i))
!_________ 6 end do
7 end
Abbreviations Used
NO not optimized
DIR directive
NCS non-concurrent-stmt
Footnote List
1: not optimized This loop contains an unoptimizable call to "FORCE".
2: not optimized This statement contains an unoptimizable call to "FORCE".
|
Since Power Fortran cannot be sure that function force has no side-effects, it cannot parallelize this loop. If the function has a side-effect—that is, if it assigns data to variables other than its arguments and result—errors would occur when it was called concurrently from multiple CPUs.
If you know that a procedure has no side-effects, you can insert an assertion to tell Power Fortran, as shown in Example 2-13.
subroutine caller(a,b,c,n)
real a(n), b(n), c(n)
external force
!*$*assert concurrent call
do i = 1,n
a(i) = force(b(i),c(i))
end do
end
|
With this modification, Power Fortran parallelizes the loop as shown in Example 2-14.
Footnotes Actions DO Loops Line
DIR 1 # 1 "pfa4.f90"
1 subroutine caller(a,b,c,n)
2 real a(n), b(n), c(n)
3 external force
DIR 4 !*$*assert concurrent call
C +--------- 5 do i = 1,n
* 6 a(i) = force(b(i),c(i))
*_________ 7 end do
8 end
Abbreviations Used
DIR directive
C concurrentized
|
There is a further subtlety in using procedure calls in parallelized loops. Power Fortran does not examine the code of the external function (unless you use the optimization of procedure inlining, discussed in the MIPSpro Fortran 90 Programmer's Guide). As a result, Power Fortran cannot know whether the external procedure modifies any of its arguments. Code with this ambiguity is shown in Example 2-15.
subroutine tricky (a,b,c,n,m)
real a(*), b(*)
external my_function
!*$*assert concurrent call
do i = 1, n
a(i) = my_function (b(i), m)
b(i) = a(i) + m
end do
end
|
The question is, does my_function modify its argument m? If it only reads m, there can be a single copy of m used by all concurrent instances of the loop. If the function assigns to m, each loop instance needs its own copy of the variable. These two modes, called SHARE and LOCAL respectively, are discussed in Chapter 7 of the MIPSpro Fortran 90 Programmer's Guide.
Given Example 2-15, Power Fortran assumes that m is read-only, and gives it SHARE status. This can be seen when the transformed program is included in the listing (-listoptions=loi), as shown in Example 2-16.
DIR 4 C*$* assertconcurrentcall
I 5 C$PAR parallel if (N .gt. 16) shared (N,A,B,M) local (I)
I 5 CSGI$ startloop 3
I 5 C$PAR pdo
LM I C +--------- 5 do 2 I=1,N
* 6 A(I) = real (MY_FUNCTION (B(I),M))
* 7 B(I) = A(I) + real (M)
I *_________ 8 2 continue
I 8 C$PAR end pdo nowait
I 8 CSGI$ endloop 3
I 8 C$PAR end parallel
|
The C$PAR clause shared(N,A,B,M) in Example 2-16 specifies that single copies of those variables will be accessed concurrently from all instances of the concurrent loop. If in fact my_function changes the value of m, then this is incorrect. If this is the case, you can give Power Fortran the hint it needs by adding a visible assignment to m prior to the procedure call, as shown in Example 2-17.
subroutine tricky (a,b,c,n,m)
real a(*), b(*)
external my_function
!*$*assert concurrent call
do i = 1, n
m = 0 ! force M to be LOCAL
a(i) = my_function (b(i), m)
b(i) = a(i) + m
end do
end
|
Since Power Fortran sees an assignment to m within the loop, it declares m to be LOCAL (without regard to its use as a parameter). This is an example of how it is sometimes necessary to put extra statements in a loop in order to get the best speed from a parallel program.
| Note: If my_function both reads the input value of m and writes a new value, the trick in Example 2-17 will not work. You would have to find an assignment to m that preserves its value in each iteration of the loop. Unfortunately, most such non-assignments, such as m=m*1, are likely to be deleted as unnecessary by the scalar optimization phase of Power Fortran! |
A reduction produces a single value from a set of values. The subroutine in Example 2-18 calculates the sum of products of two arrays.
subroutine sum_prods(a,b,n,sum)
real a(n), b(n), sum
sum = 0.0
do i = 1,n
sum = sum + a(i)*b(i)
end do
end
|
With Example 2-18 as input, Power Fortran produces the listing in Example 2-19.
1 subroutine sum_prods(a,b,n,sum)
2 real a(n), b(n), sum
3 sum = 0.0
+--------- 4 do i = 1,n
1 DD ! 5 sum = sum + a(i)*b(i)
!_________ 6 end do
7 end
Abbreviations Used
DD data dependence
|
Because different iterations of the loop read and write the variable sum, Power Fortran detects a dependence. However, reduction is a special kind of data dependence. Because sum only accumulates a total, you can accumulate subtotals in parallel and then combine the subtotals at the end. (This general technique for dealing with reduction is discussed in Chapter 7 of the MIPSpro Fortran 90 Programmer's Guide.)
In fact, Power Fortran can recognize the four most common reductions—sum, product, min and max—and parallelize them. It does not do this automatically because, since the parallel version of the code adds the elements together in a different order than the single-process version, the round-off errors can accumulate differently in a parallelized reduction, producing a different answer depending on the number of CPUs that execute the code. In fact, if you use the dynamic scheduling option, the answer might vary slightly from one run of the program to the next, even if you use the same number of processes on the same machine.
Most applications can safely ignore this variation in round-off error. If your application is not affected, you can direct Power Fortran to use parallel reduction using either the C*$* ROUNDOFF(2) directive or the driver option –WK,–roundoff=2. The resulting listing is shown in Example 2-20.
DIR 1 # 1 "pfa6.f90"
1 subroutine sum_prods(a,b,n,sum)
2 real a(n), b(n), sum
3 sum = 0.0
DIR 4 !*$*roundoff(2)
C +--------- 5 do i = 1,n
SO * 6 sum = sum + a(i)*b(i)
*_________ 7 end do
8 end
Abbreviations Used
SO scalar optimization
DIR directive
C concurrentized
|
The round-off error produced by a parallel reduction operation is not necessarily worse than the round-off error seen in the serial version. It is simply different. If your original application was not affected by round-off error, there is no reason to worry about it in the parallel version. However, if your application takes special steps to reduce round off (for example, adding the numbers together in order from smallest absolute value to largest), then you should not use parallel reductions.
Table 2-6 shows the types of reductions Power Fortran supports.
Table 2-6. Power Fortran Reductions
Type | Operator | Example |
|---|---|---|
Sum | + | sum = sum + expression |
Product | * | prod = prod * expression |
Min | min( ) | least = min(least, expression) |
Max | max( ) | most = max(most, expression) |
All these reductions are under the control of the –roundoff driver option and C*$*ROUNDOFF directive, even though technically the min and max reductions do not involve round-off problems.