The primary recommendation for data distribution is to organize algorithm inputs and outputs on separate SRAM banks. In other words, if bank 0 is used for input data, it should not also be used for output data and vice versa (by splitting the SRAM into two logical halves, for example). The motivation for this guideline comes from the fact that when an algorithm accesses a particular bank's read or write port, it blocks access to the DMA engine that wants to unload finished results or load new operands.

To avoid multiple arbitration cycles that add to read and write latency, the algorithm is automatically given access to the ports while the algorithm is active.

In order for the hardware accelerated algorithm to run efficiently on large data sets, it is recommended to overlap data loading and unloading with algorithm execution. To do this successfully, the algorithm designer needs to start with an SRAM layout of algorithm operands and results that allows each bank to be simultaneously accessed by the Algorithm for one direction (read or write) and a DMA engine for the other direction (write or read).

Clock cycle based stepping means that the step size is one clk cycle. This method is easily implemented in RTL-based algorithms by tying the step_flag output to one (logic high). The step counter used by the debugger is 16 bits, so the maximum number of clock cycles that can be stepped in one debugger command is 2^16-1 = 65,535.

Note that since the Algorithm Block cannot detect when clk has stopped, the effect of stepping is transparent to the Algorithm Block.

An example of this mode is shown in Figure 3-6.

Figure 3-6. Clock Cycle Stepping Mode Example

Another approach for implementing debugging is to assert step_flag at points of interest rather than every clock cycle, which makes the step size a variable number of clock cycles. One example of this method would be to use step_flag as an output of the last state of the FSM. Another example would be for the user to put in a “trigger” for when an internal counter or state machine reaches a specific value (with an indeterminate number of clock cycle steps in between). In this case, step_flag is tied to the trigger so that the algorithm can break at a designated point.

The ad hoc nature of this approach requires the Algorithm to define and notify Core Services of step boundaries with the step_flag signal. The clk signal will not stop toggling during the same clock cycle that the step_flag signal is asserted; it will turn off on the following clock cycle. See the timing diagram in Figure 3-7.

Figure 3-7. Variable Step Size Mode Example

Figure 3-8 shows single and back-to-back write commands.

Figure 3-8. Single, and Multiple Write Commands

The process of using a read port involves the following steps (example given for SRAM0 alone):

Figure 3-9 and Figure 3-10 show single and back-to-back read commands.

Figure 3-9. Single Read Transaction

Figure 3-10. Multiple Read Transaction

Many applications targeted for hardware acceleration have large input and output data sets that will need to be segmented to fit subsets into the RASC-brick's external SRAM banks at any given time (a total of 32MB are available). Certain applications have large input data sets that have the same processing performed on subsets and only require that new input data be available in order to maintain continuous processing. These requirements bring up the notion of streaming a large data set through an algorithm. Streaming data suggests a continuous and parallel flow of data to and from the algorithm. Streaming is to sequentially load data, execute, and unload a block of data in a loop one at a time.

In order for the hardware accelerated algorithm to run efficiently on large data sets, it is recommended to overlap data loading and unloading with algorithm execution. To do this successfully, start with an SRAM layout of algorithm operands and results that complies with the recommendations for memory distribution. The input and output data needs to be segmented into at least two segments per SRAM bank so that the Algorithm Block can execute on one segment while the next data segment is loaded by the Read DMA Engine and the previous output data segment is unloaded by the Write DMA Engine (ping-pong buffer effect). The SRAM bank can be segmented into any number of segments, as the algorithm and application designers best see fit.

---

Note: The term streaming as used in this section differs from the conventional definition of streaming.

---

In order to support streaming, an algorithm should allow the upper bit(s) of its read and write SRAM addresses be programmable variable via the sram*_offset inputs. The offset inputs come from an internal FPGA register within the Core Services Block, accessible by the software layer. The offset inputs are 10 bits each, and can map to bits [19:10] of the corresponding SRAM address. Only the bits that correspond to the segment offset are used for a particular algorithm/application. For example, if the segment size is 32 KB, which leads to 512 segments in the 16MB bank, only the upper 9 of the 10 offset bits are used. Example Register Transfer Level (RTL) code for this configuration is shown below:

reg [8:0] rd_segment_addr_sram0;
reg [10:0] rd_laddr_sram0;
wire [19:0] rd_addr_sram0;

// Read pointer
always @(posedge clk)
begin
	 	 // Fixed upper address bits [19:11] per iteration
	 if (rst) rd_segment_addr_sram0 <= alg_sram0_offset[9:1];

// Counter for lower address bits [10:0]
	 	if (rst)
	 	 	 rd_laddr_sram0 <= 11'h000;
else if (rd_advance_en)
	 	 	 rd_laddr_sram0 <= rd_addr_sram0 + 1'b1;
end

assign rd_addr_sram0 = {rd_segment_addr_sram0, rd_laddr_sram0};

The algorithm must define a legal segment size between the minimum and maximum allowable sizes, and only operate on and generate one segment worth of data per iteration.

Special extractor directives are required to pass information about the algorithm's data layout and streaming capabilities to the software layer. The software layer requires a declaration of the input and output data arrays on each SRAM bank, with attributes size, type, and buffer-ability defined. The declarations are provided as extractor directives, or comments in the Verilog or VHDL source code. The following example comments are used to declare two 16KB input data arrays located on SRAM 0, and one non-streaming input parameter array, also located on SRAM 0:

##Array name
##	 	 	 	# of elements in array
##	 	 	 	 	 	 	 	 	 Bit width of element
##	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 SRAM location
##	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 Byte offset (within given SRAM)
##	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 Direction
##	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 Type
##	 	 		 	 	 	 	 	 	 	 	 	 	 	 		 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 		 Stream flag
// extractor SRAM:input_a 2048 64 sram[0] 0x000000 in unsigned stream
// extractor SRAM:input_b 2048 64 sram[0] 0x004000 in unsigned stream
// extractor SRAM:param_i 512 	 64 sram[0] 0xffc000 in unsigned fixed

For arrays that are defined as buffered, the byte offset provided in the extractor comments is used to establish the data placement within a particular segment. In the declaration, the byte offset is given at the lowest segment address. For fixed arrays, the byte offset is the absolute placement of the data within the SRAM.

Further details on extractor comments can be found in Chapter 5, “RASC Algorithm FPGA Implementation Guide”

Software uses configuration information derived from extractor directives to move the current active segment on each run of the algorithm. Software changes the values of sram0_offset[9:0] and sram1_offset[9:0] and for each run of the algorithm and similarly moves the DMA engines' input and output memory regions based on the current active segment.

The method used to pass variable parameters depends on the size and number of the required parameters. For a small number of 1-8 byte-sized parameters, the Algorithm Block can associate parameters with up to eight Algorithm Defined Registers. The Algorithm Defined Registers are eight optional registers that exist within the SSP memory mapped register region whose current values are inputs to the Algorithm Block (alg_def_reg0[63:0] - alg_def_reg7[63:0]). The Algorithm Block can assign reset (default) values for the parameters by tying the output signals alg_def_reg0_rst_val[63:0] - alg_def_reg7_rst_val[63:0], and allow the host application to change them. The algorithm can also use the alg_def_reg_rst_en signal to select whether there should be a reset value, or if the parameter should be unaffected by reset.

One example is shown here is, as follows:

always @(posedge clk)
	 	 begin
	 	 	 	 match_val1 <= alg_def_reg0[15:0];
	 	 	 	 match_val2 <= alg_def_reg0[31:16];
	 	 end

The algorithm passes information about the mapping of parameters to Algorithm Defined Registers to the software Abstraction Layer by way of the algorithm configuration file. The configuration file and its creation is discussed in detail in Chapter 5, “RASC Algorithm FPGA Implementation Guide”

To summarize here, comments about the parameters are added to the algorithm source code, similar to synthesis directives. Software needs to know what the parameter's descriptor is, how many bits it comprises, the data type of the parameter, and which Algorithm Defined Register it is associated with. An extractor tool parses these comments and builds up a table of algorithm specific information for use by the Abstraction Layer. Therefore, in order for these parameters to be recognized by the Abstraction Layer and used by the host application, extractor directives need to be added to the source code to indicate parameter mapping.

A simple example is provided below. In this example, the parameter match_val is 32-bits wide, unsigned, and mapped to alg_def_reg0, bits [63:32].

// extractor REG_IN:<parameter_name> <bit width> <type:signed/unsigned> alg_def_reg[<reg_num>][<bit range>]

// extractor REG_IN:match_val 32 u alg_def_reg[0][63:32]

When an algorithm requires larger fixed parameters, portions of the SRAM banks can be used to hold the parameter data. This portion of the SRAM needs to be reserved for parameter data and kept unused by input data, so parameters need to be considered in the initial memory allocation decisions. Just as with small parameters, the mapping of parameter data to SRAM addresses is specified with extractor comments. The template and an example is provided below; further details are in the “Adding Extractor Directives to the Source Code” in Chapter 5. In the provided example, a 1024-element parameter matrix (8KB) is mapped to the upper 8KB of SRAM0, which starts at address 0xFFE000. The type is unsigned and the array is fixed, which denotes that it is a parameter array and not as variable as an input data array (the other option is “stream”).

// extractor SRAM:<parameter array name> <number of elements in array> <bit width of elements> <sram bank> <offset byte address into sram> in <data type of array> fixed

// extractor SRAM:param_matrix0 1024 64 sram[0] 0xFFE000 in u fixed

Another use of declaring a fixed array in one of the SRAMs could be for a dedicated scratch pad space. The only drawback to using SRAM memory for scratch pad space is that an algorithm normally writes and then reads back scratch pad data. This usage model violates the streaming rule requiring an algorithm to dedicating each SRAM bank for either inputs or outputs. If you have a free SRAM bank that you do not need for inputs or outputs, this violation can be avoided and the streaming model can be maintained. If you have a free SRAM that is not being used for anything else, then you do not even have to add an extractor directive. An extractor directive is necessary if the SRAM bank is being used for other purposes so that software does not overwrite your scratch pad space. An extractor directive is also necessary to be able to access the scratch pad space from the debugger (reads and writes), so in general, an extractor directive is recommended.

Note that if you violate the streaming model for SRAM direction allocation, data will not be corrupted but the benefit of streaming will not occur because data transfer and algorithm execution cannot be overlapped. A template and an example is provided below for writing an extractor comment for a scratch pad space.

// extractor SRAM:<scratch_pad_array_name> 
                  <number of elements in array> 
                  <bit width of elements> 
                  <sram bank> 
                  <offset byte address into sram>
                  inout
                  <data type of array>
                  <signed / unsigned>
                  fixed

// extractor SRAM:scratch1 1024 64 sram[2] 0x000000 inout u fixed

The core clock has a fixed clock rate of 200 MHz. It is the main clock used within the Core Services Block. It is derived from the input control clock on the Scalable System Port (SSP) interface. The input clock is used as the clock input to the core clock digital clock manager (DCM) module. The clk0 output of the DCM is driven onto a low-skew global clock buffer (BUFG) and from there is routed to core_clk domain registers as well as the feedback of the DCM for fine phase adjustments. In the place and route report, this clock is labelled core_clk.

The algorithm clock rate is selectable through the use of macro definitions. Speeds of 50, 66, 100, and 200 MHz can be selected. For speeds slower than 200 MHz, the DCM clock divider in the Xilinx Virtex 4 FPGA is used to create the specified clock signal. Figure 3-12 shows a block diagram of the logic used to create the clock signals.

Macro definitions are used to select between the clk0 and clkdv outputs of the DCM to drive the BUFGCE clock buffer, which in turn drives the clk signal in the Algorithm Block via the alg_clk signal.

The signal alg_clk_enable (not shown) generated by Core Services gates alg_clk. By gating the clock, the signal toggles only when the Algorithm Block is active. In the place and route clock report, this clock is labelled alg_clk.

When clkdv is used to drive the algorithm clock, the phase relationship between the core clock and the algorithm clock is determined by the Virtex 4 parameter CLKOUT_PHASE, which specifies the phase relationship between the DCM outputs of the same DCM. For Virtex 4 FPGAs, this parameters is specified as +/- 140 ps. Although the Xilinx timing tools do not take the CLKOUT_PHASE into account directly during analysis, an additional 140 ps has been added as input clock jitter to force the tools to correctly check paths crossing the core_clk and alg_clk domains. Any phase difference that is derived from the varying routes between the DCM outputs and the BUFG / BUFGCE elements as well as the clock tree fanouts are automatically considered by the Xilinx timing tools.

Core clock domain signals in the Core Services Block that communicate with the Algorithm Block, which is entirely in the alg_clk domain, have been designed to transition on the rising edge of alg_clk, even when alg_clk is run slower than the rate of core_clk.

Figure 3-12. Core Clock and Algorithm Clock Source

Besides the core clock domain, which is equivalent to the Scalable System Port (SSP) control group domain, there are four data group clock domains within the SSP source synchronous receiver and transmitter logic. These four groups have a determined phase relationship between one another: each group is separated by a nominal 250 ps shift to reduce simultaneous switching noise on the SSP interface. In the place and route clock report, these clocks are labelled int_grp1_clk - int_grp4_clk.

The QDR-II SRAM module interfaces, a part of Core Services, uses five additional clock domains: one receive clock for each external SRAM (for a total of four, one for each physical SRAM component), and one common data transmit clock, which is a quarter clock cycle early relative to the core_clk. In the place and route clock report, these clocks are labelled bank0_sram_rcv_clk, bank1_sram_rcv_clk, bank2_sram_rcv_clk, bank3_sram_rcv_clk, and core_clk270.

The algorithm clock rate can be set at 50, 66, 100, or 200 MHz.

The following is a portion from the above example include file that selects the clock speed. This shows the four macros used to specify an algorithm clock speed.

////////////////////////////////////////////
//    Specify clock speed of algorithm    //
////////////////////////////////////////////
// Only one of the below four should be uncommented

// For 200 MHz
  `define alg_clk_5ns

// For 100 MHz
//  `define alg_clk_10ns

// For  66 MHz
//  `define alg_clk_15ns

// For  50 MHz
//  `define alg_clk_20ns

One and only one of the above four macros must be defined. Here the macro to set the algorithm clock rate to 200 MHz is defined and uncommented whereas the other macros are left undefined.

In order for the Algorithm Block to have access to a given SRAM port, the corresponding macro must be defined in alg.h.

The following portion from the include file enables access to SRAM ports.

////////////////////////////////////////////
//  Specify SRAM ports for algorithm use  //
////////////////////////////////////////////

`define alg_uses_sram0_rd
//`define alg_uses_sram0_wr
//`define alg_uses_sram1_rd
`define alg_uses_sram1_wr

All, some, or none of these macros can be defined. In this example, the read port for SRAM bank 0 and the write port for SRAM bank 1 are enabled, while the write port for SRAM bank 0 and the read port for SRAM bank 1 cannot be used by the Algorithm Block.

If the algorithm is to buffer a large data set, the macros for the read port of one SRAM bank and the write port for the other must be undefined. Otherwise, ports will be inaccessible to the DMA engines while the Algorithm Block is active, and therefore, DMA and the Algorithm Block will not be able to operate concurrently.

Compiling the sample test bench is done using the Makefile provided. In order to compile the sample testbench including the SSP Stub and the algorithm Core Services logic, an algorithm must be specified (See the following note).

---

Note: The Makefile in the sample_tb directory uses the $ALG_DIR environment variable. This defaults to $RASC/examples though it can be modified by the user. The design files of the algorithm you specify must be in a directory under the $ALG_DIR path.

---

The algorithm you are building is specified on the command line. To compile the design with your algorithm, change directory to $RASC/dv/sample_tb and enter:

% make ALG=<your_algorithm>

where <your_algorithm> is the directory name where the algorithm design files are. When no algorithm is specified, the default is ALG=alg_simple_v.

To remove older compiled copies of the testbench, type:

% make clean

To run a diagnostic on your algorithm, call the Makefile in the sample_tb directory using the “run” target and specifying which diag to run. The following is the usage and options of the “run” target:

% make run DIAG=diag_filename ALG=your_agorithm
SRAM0_IN=sram0_input_file SRAM1_IN=sram1_input_file
SRAM2_IN=sram2_input_file SRAM3_IN=sram3_input_file
SRAM0_OUT=sram0_output_file SRAM1_OUT=sram1_output_file
SRAM2_OUT=sram2_output_file SRAM3_OUT=sram3_output_file

The diag_file specifies the diagnostic to be run and should be relative to the current directory. Again, the algorithm must be specified using the ALG=your_algorithm command line option. If none is specified, the runtime command uses same default as above (ALG=alg_simple_v). Specifying ALG this way allows the user to reuse the same diagnostic for multiple algorithms. The contents of each SRAM at the end of simulation will be dumped into .dat files that can be user-specified. If they are not specified, they default to:

init_sram0_good_parity.dat
init_sram1_good_parity.dat
init_sram2_good_parity.dat
init_sram3_good_parity.dat
final_sram0.dat
final_sram1.dat
final_sram2.dat
final_sram3.dat

Note that there are four input and four output SRAM data files while the design is implemented for two logical SRAMs. Each of the logical SRAMs is implemented as two separate physical SRAMs in the sample testbench. The sram0* and sram1* files correspond to the first logical SRAM while sram2* and sram3* correspond to the second logical SRAM.

By specifying the SRAM input and output files the user can skip the DMA process for quick verification of the algorithm. This shortens the diagnostic run time, makes for less complex diagnostics, and allows the user to ignore core services as it has already been verified by SGI. The option of utilizing the DMA engines in simulation is included for completeness but should not be necessary for typical algorithm verification.

As the diagnostic runs, it will output status to the screen and to an output file named <diag_filename>.<your_algorithm>.run.log. When the stub receives an unexpected packet, it will output the following information in order: the command for the next expected packet, SSP fields of the expected packet, the command translation (if one exists) for the received packet, and the SSP fields of the received packet. This log file will appear in the same directory in which that the diagnostic is located.

Table 3-7 shows a summary of the algorithms, diagnostics, and commands provided with the sample testbench.

Each time a diagnostic is run, a file named vcdplus.vpd is generated in the sample_tb directory. This file can be input to Virsim for viewing the waveform. Since this file is generally large, it is overwritten for each diagnostic run. To save the waveform for a given diagnostic, copy the corresponding vcdplus.vpd file to a new name.

To view the waveform saved in the vcdplus.vpd file, use the following command:

% vcs -RPP vcdplus.vpd

A sample configuration file sample_tb/basic.cfg is provided in the for use when viewing waveforms in Virsim. It contains a limited number of relevant signals on the SSP interface, SRAM interfaces, and inside the design. Figure 3-13 shows a sample vcdplus.vpd waveform in Virsim.

Figure 3-13. Sample vcdplus.vpd Waveform in Virsim