| | |

Quality Assurance

Many people know plusFORT mainly for SPAG, the restructuring tool which unscrambles spaghetti code. However, the plusFORT quality assurance facilities are arguably of even greater importance to most users.

Fortran Quality Assurance using plusFORT

plusFORT Version 6 is unique among QA tools in offering three distinct and complementary approaches to the problem of software quality assurance. Working together these three approaches have far more impact than any one could by itself.

Static Analysis.

plusFORT analyses your source code WITHOUT executing it, and detects errors on a program-wide basis. Every CALL and FUNCTION reference is thoroughly checked against the correponding SUBROUTINE or FUNCTION definition, and the consistency of COMMON block definitions is verified. The plusFORT static analyser is able to detect innumerable errors and anomalies, such as COMMON variables which are used but never set, and PARAMETERs defined with different values in different subprograms.

plusFORT also computes complexity metrics for each subprogram. These figures may help to indicate where rationalisation work should be concentrated.

A big advantage of static analysis is that it can check all your code without regard to whether or not it is actually executed in a particular run. On the other hand, static analysis cannot take account of subtleties in program control flow, or the varying nature on input data. Neither can it cannot monitor the status of individual array elements. Arrays are treated as amorphous blobs of data.
Dynamic Analysis.

plusFORT creates a test version of your program which detects any use of an undefined variable or array element, and logs it to a file. In other respects, the test program behaves exactly like the original. The use of undefined data is a feature of a large proportion of all program bugs, and dynamic analysis is, in consequence, an extremely powerful QA tool.

Unlike static analysis, dynamic analysis takes account of program control flow, works with real data, and can monitor individual bytes if required. However it only validates the code which is executed in a particular run. This makes it an almost precise complement of static analysis.
Test Coverage and Hot-Spot Analysis.

A reasonable goal for a software test suite is to ensure that every line of source code is executed at least once. The plusFORT coverage analysis facility allows users to monitor progress against this goal, as well as identifying "hot-spots" - the sections of code which are executed most frequently, and which have most effect on program execution time.

Top

Global Static Analysis

GXCHK is the plusFORT global cross check tool. GXCHK views data usage from a global perspective, and detects errors and anomalies that compilers and other tools miss. Examples are:

Subprogram argument mismatch or misuse (e.g. constant actual argument is illegally modified by subprogram).
COMMON variables assigned a value but never used, or used but never assigned a value.
Globally unused COMMON variables, COMMON blocks, PARAMETERs, & INCLUDE files.
Inconsistent COMMON block definitions.
COMMON variable name appears in other contexts (e.g. as a local variable, or in a different COMMON).

An unusual feature of GXCHK is its ability to analyse local and global data together. For example, if a local variable in subprogram A has the same name as a COMMON variable in subprogram B, it could be that the programmer has omitted the relevant COMMON statement from subprogram A. GXCHK, unlike other static analysers spots this problem. GXCHK also produces a call tree, and concise but comprehensive charts showing where and how each symbol is used. Handy aggregated reports are produced for COMMON blocks and INCLUDE files.

Interface specifications for every subprogram, showing calls in and out, dummy arguments, COMMON variables usage etc., can also be produced.

GXCHK operates in a "compile & link" style which allows reports to be updated with minimal source code analysis. AUTOMAKE can be used to automate the process.

Interactive Static Analysis

The GXCHK database can be queried interactively. This allows users instant access to information which is vital to program development and maintenance. For example, a programmer might set up a static analysis query window in a corner of the screen, and use it to find the consequences of a change to a COMMON block.

GXCHK Diagnostic Reports

COMMON variable is never referenced          CALCST in /REACTN/
COMMON variable name used in other contexts  CC
COMMON var is given a value, but never used  CC in /TEMP/
    .-----------------------------------------------------------------------------------
    |CC       DOUB COMMON     array with 2 dims pos   1 in /TEMP/    in +COMMONS
    |CC       DOUB local      variable          DEFINED in PRTDRC    in  PRTDRC.FOR
    '-----------------------------------------------------------------------------------
COMMON var is used, but never given a value  CG in /TEMP/
COMMON variable name used in other contexts  CNORML
    .-----------------------------------------------------------------------------------
    |CNORML   DOUB COMMON     array with 1 dims pos   1 in /VECTOR/  in FORCE    in  FORCE.FOR
    |CNORML   DOUB dummy arg  array with 1 dims pos   3 in FREQCY     in  FREQCY.FOR
    '-----------------------------------------------------------------------------------

COMMON Variable Usage Report

ANEXT ......... changed by SEARCH
                   used by POWSQ    SEARCH
AQM ........... changed by AA0001   CALPAR
                   used by MOLDAT
AQPM3 ......... changed by AA0001
                   used by MOLDAT
ATHEAT ........ changed by AM1      MOLDAT
                   used by AM1      COMPFG   ITER     MOLDAT
ATMASS ........ changed by GETGEO
                   used by AXIS     DRC      FMAT     FRAME
                           FREQCY   PRTDRC   WRITE
BDD ........... changed by ** NOTHING **
                   used by DELRI
BETA3 ......... changed by AA0001
                   used by ANALYT   H1ELEC
BETAD ......... changed by MOLDAT   UPDATE
                   used by CALPAR   H1ELEC   MOLDAT
BETAP ......... changed by AA0001   MOLDAT   UPDATE
                   used by ANALYT   CALPAR   H1ELEC   MOLDAT

A GXCHK Call Tree

0001: MAIN
0002: |--AM1
0003: |  |--MOLDAT
0004: |  |  |--GMETRY
0005: |  |  |  '--GEOUT
0006: |  |  |     '--XYZINT
0007: |  |  |        |--BANGLE
0008: |  |  |        |--DIHED
0009: |  |  |        |  '--DANG
0010: |  |  |        '-XYZGEO
0011: |  |  |          |--BANGLE
0012: |  |  |          '---DIHED      > 0008
0013: |  |  |--READA
0014: |  |  |  '--DIGIT
0015: |  |  '--VECPRT
0016: |  |--READA      > 0013
0017: |  '--UPDATE
0018: |-CMNAM      <- unresolved 0019: |-COMPFG

Top

Dynamic Analysis

If a bug is defined as "an incorrect value in a storage location"
then there is a large subset (perhaps 30%) which can be defined as
"an undefined value in a storage location", and many of the remainder
may cause that condition as a knock-on effect.

The Dynamic Analysis option of plusFORT is a tool for diagnosing these errors at run-time. Calls to probe routines are inserted in the source code before any operation which depends on the value of a data item, and the program is compiled and linked in the normal way. The executable code appears to the user to operate in exactly the same way as the original, but if a probe detects an undefined data item, it writes details to a log-file for later analysis. Source code for the probe routines is supplied.

A static analyser such as GXCHK can detect a few of these errors, but the majority can only be detected at run-time. In fact static and dynamic analysis complement each other well - each excels where the other is deficient.

Feature	Static Analysis	Dynamic Analysis
Checks code whether or not it executes	Yes	No
Checks for unsafe/questionable source code	Yes	No
Identifies used before set error	Sometimes	Yes
Checks status of individual array elements	No	Yes
Handles dependence on external data	No	Yes
Effective with dummy args, EQUIVALENCE	poor	Yes

Dynamic Analysis - Worked Example

Original Code

      SUBROUTINE QKNUM(Ival,Pos)
      INTEGER Ival , Pos , vals(50) , ios
      READ(11,*,IOSTAT=ios) vals
      IF ( ios.EQ.0 ) THEN
         DO Pos = 1 , 50
            IF ( Ival.EQ.vals(Pos) ) GOTO 100
         ENDDO
      ENDIF
      Pos = 0
 100  END

QKNUM  reads  50 numbers, and returns the position (Pos) of the first
with value Ival.

If the data file contains "2,9,,11,20*44,26*", the READ statement leaves some elements of vals undefined, and QKNUM behaves unpredictably.

Code with Dynamic Analysis Probes

      SUBROUTINE QKNUM(Ival,Pos)
      INTEGER Ival , Pos , vals(50) , ios
      CALL SB$ENT('QKNUM','DYNBEF.FOR')
      CALL UD$I4(IOS)                     <-- set variable IOS and array VALS to undefined CALL UD$AI4(50,VALS) READ (11,*,IOSTAT="ios)" vals CALL QD$I4('IOS',IOS,4) IF ( ios.EQ.0 ) THEN DO Pos="1" , 50 CALL QD$I4('VALS(POS)',VALS(POS),6) CALL QD$I4('IVAL',IVAL,6) <-- check that VALS(IPOS) and IVAL are defined before using them IF ( Ival.EQ.vals(Pos) ) GOTO 99999 ENDDO ENDIF Pos="0" 99999 CALL SB$EXI <-- calls to SEB$ENT and SB$EXI provide traceback and timing data END

Dynamic analysis excels at detecting bugs which depend on external data, or on
the use of arrays.  These characteristics defeat conventional analysers.

Sample Log-file

XTOP2(I+J) value is undefined
 subprogram    FCOMP line    838 file JASO.FOR
 subprogram    VSCAN line   4255 file JASO.FOR
 subprogram   VUTREE line   1049 file JASO.FOR
 
ICHR value is undefined
 subprogram    VSTOP line   6682 file JASO.FOR
 subprogram     JASO line   2241 file JASO.FOR

Top

Coverage Analysis

The plusFORT coverage analysis facility places probes into 
Fortran source code which allow users to monitor the effectiveness of testing.
At the end of each run, the probes update the coverage statistics for each 
source file.  This data may be analysed at any time using the CVRANAL tool.
CVRANAL identifies untested code blocks, and execution hot-spots.

In addition, CVRANAL can annotate your source code as shown below. The annotations are comments and do not affect the validity of the source code.

Fragment of CVRANAL Test Coverage Report

4 Untested code blocks in subprogram MOO in file TESTMOO.BEF
     at lines     27    32    33    62
NO   Untested code blocks in subprogram BC in file TESTMOO.BEF
NO   Untested code blocks in subprogram GETNUM in file TESTMOO.BEF
NO   Untested code blocks in subprogram INIT in file TESTMOO.BEF

Fragment of CVRANAL Execution Hot-spot Report

49682 :       IF(IGUESS(IDIG).NE.IANS(JDIG))GOTO 10
               : in s/prog BC at line    82 of TESTMOO.BEF
         16336 :       IF(IGUESS(IDIG).EQ.IANS(IDIG))GOTO 40
               : in s/prog BC at line    80 of TESTMOO.BEF
         14399 :       DO 10 JDIG=1,NDIGIT
               : in s/prog BC at line    81 of TESTMOO.BEF
         11283 : 10    IF(J.EQ.NUM(JDIG))GOTO 20
               : in s/prog GETNUM at line   104 of TESTMOO.BEF
          9039 :       GOTO 20
               : in s/prog BC at line    86 of TESTMOO.BEF

Annotated Source Code

SUBROUTINE BC(IGUESS,IANS,NBULLS,NCOWS)
      PARAMETER (NDIGIT=4,MINDIG=1,MAXDIG=9)
      DIMENSION IGUESS(NDIGIT),IANS(NDIGIT)
 
      NBULLS=0                                           !   4084
      NCOWS=0
      DO 20 IDIG=1,NDIGIT
      IF(IGUESS(IDIG).EQ.IANS(IDIG))GOTO 40              !  16336
      DO 10 JDIG=1,NDIGIT                                !  14399
      IF(IGUESS(IDIG).NE.IANS(JDIG))GOTO 10              !  49682
      NCOWS=NCOWS+1                                      !   5360
      GOTO 20
10    CONTINUE
      GOTO 20                                            !   9039
40    NBULLS=NBULLS+1                                    !   1937
20    CONTINUE
      RETURN                                             !   4084
      END

  for further information

Top

Programs for Programmers