MATRIX - Section 52 The B34S programming language capability is called with the command b34sexec matrix; /$ commands here b34srun; Example: b34sexec matrix; x=matrix(2,2:22. 33. 44. .02); ix=inv(x); test=ix*x; call print(x,ix,test); b34srun; The MATRIX command provides a 4th generation language to process selected calculations from B34S procedures and the ability to further process data. Analysis is supported for real*8, real*16, complex*16 and complex*32 data. Other data types such as character*1, character*8, real*4 and integer*4 are supported. Calculation of the inverse of a real*4 matrix, while supported, is not recommend due to accuracy loss. High resolution graphics are available on all currently supported platforms (Windows, RS/6000, Sun, Linux) and batch and interactive operation is supported. Many string operations are available for character*8 and character*1 data. The MATRIX facility supports user PROGRAMS, SUBROUTINES and FUNCTIONS. SUBROUTINES and FUNCTIONS have their own address space. Variables are built using object oriented programming with analytic statements such as: y = x*2.; where x is a variable that could be a matrix, 2D array, 1D array, vector or a scalar. The use of the MATRIX command FORMULA and the SOLVE subroutine allows recursive solution of an analytic statement over a range of index values. This facility speeds up calculations that would have had to use do loops which have substantial overhead. For intensely recursive calculations, the user can call Fortran routines from within a MATRIX command program or subroutine. The MATRIX command recognizes variables of KIND complex*16, real*8, real*4, real*16, complex*32, integer*4, character*1 and character*8. Due to possible accuray loss only array math is supported for integer*4 objects. While the inv(real4object) is supported, due to accuracy loss this is not recommended. The KLASS of an object determines how it is processed. KLASS types are scalar, 1D array, 2D array, vector and 2D matrix. For example the assingment statement y=2.0; sets y to be a real*8 scalar, while y=2; sets y to be a integer*4 scalar. If the commands x=33.; y=2; test=x*y; are given there will be a mixed-mode error message since the MATRIX command processor does not know whether test should be integer*4 or real*8. To create an integer*4 from a real*8 use testint=idint(x)*y; while to create a real*8 from an integer*4 variable y use testr8 =x*dfloat(y); Where possible Fortran names have been used for built-in functions to reduce the learning curve. To create a 2D array use xarray=array(2,2: 1., 2., 3., 4.); To create a 2D matrix use xmatrix=matrix(2,2:1., 2., 3., 4.); or xmatrix=mfam(xarray); There are a number of built in functions to convert the KIND and KLASS. Key ones are: SFAM( ) Create a scalar family. MFAM( ) Convert an array to a matrix. AFAM( ) Convert a matrix to an array. VFAM( ) Convert a 1D array to a 1D vector. TO_RMATRIX( ) Convert Object to Row-Matrix TO_CMATRIX( ) Convert Object to Col-Matrix TO_RARRAY( ) Convert Object to Row-Array TO_CARRAY( ) Convert Object to Col-Matrix TO_VECTOR( ) Convert Object to Vector TO_ARRAY Convert Object to Array DFLOAT( ) Convert integer to real*8. IDINT( ) Convert real*8 to integer. IQINT( ) Convert real*16 to integer. IDNINT( ) Nearest integer from real*8. IQNINT( ) Nearest integer from real*16 REAL( ) Obtain real*8 part of a complex*16 data type. QREAL( ) Obtain real*16 part of complex*32 data type. IMAG( ) Obtain the real*8 imaginary part of a complex*16 data type. QIMAG( ) Obtain the real*16 imaginary part of a complex*32 data type. COMPLEX( , ) Build a complex*16 variable from two real*8 variables. QCOMPLEX( , ) Build a complex*32 variable from two real*16 variables. QNINT( ) Real*16 representation of nearest whole number. DNINT( ) Real*8 representation of nearest whole number. R8TOR16( ) Real*8 to real*16. R16TOR8( ) Real*16 to real*8. C16TOC32( ) Complex*16 to complex*32. C32TOC16( ) Complex*32 to complex*16. KINDAS(x,yy) Set a variable the kind of x, but value of yy. More detail on these features will be given below. The B34S MATRIX language is very close to the Speakeasy language. In MATLAB all 2D and 1D objects are assumed to be what B34S calls the MATRIX KLASS. Assume x and y are 3 by 3 MATRIX objects in B34S and MATLAB. In MATLAB for an element by element operation the command is newobj=x.*y; In B34S and Speakeasy the command for the same operation is: newobj=afam(x)*afam(y); If it is desired to place this back in a matrix: newobj=mfam(afam(x)*afam(y)); Unlike MATLAB, B34S and Speakeasy are not case sensitive. Assume xarray is an array and xmatrix is a matrix. xxarray =2.+xarray; xxmatrix=2.+xmatrix; Speakeasy and B34S work the same. Here 2. is added to all elements in xarray and only the diagonal elements in xmatrix. In MATLAB 2+x adds 2 to all elements of the matrix. The development of the MATRIX facility has been influenced closely by Speakeasy. Wherever possible, the same command language has been used to facilitate transfer of data to Speakeasy for further processing. The MATRIX facility is NOT designed to run interactively but commands can be given interactively in the MANUAL mode. In this mode scripts can be edited and submitted. Output is written to the b34s.out file and error messages are displayed in both the b34s.out and b34s.log files. Under the Display Manager, the user can scroll the output files and run the MATRIX command before and after other b34s commands. The objective of the MATRIX facility is to give the user access to a powerful object oriented programming language so that custom calculations can be made. Of major interest is providing the ability to estimate complex nonlinear least squares and maximum likelihood models. Such models, which are specified in B34S MATRIX command programs, can be solved using either other B34S subroutines or with the MATRIX nonlinear commands NLLSQ, NL2SOL, MAXF1, MAXF2, MAXF3, CMAXF1, CMAXF2 and CMAXF3. While the use of B34S subroutines would give the user TOTAL control of the estimation process, speed would be given up. In addition to specification of the model, in the B34S MATRIX language it is also possible to write the model in a Fortran or C program and call this user program from within a B34S MATRIX program. For recursive systems where it is near impossible to vectorize the calculation, this is may be the best way to proceed. The B34S nonlinear solvers are based on time tested routines. The objective of the b34s implementation is to facilitate their use in a way whereby there is full knowledge of just what is being calculated. The design of the MATRIX facility allows other libraries of routines to be accessed from C or Fortran to provide other alternatives. The MATRIX nonlinear commands give the user complete control of the form of the estimated model which is specified in a MATRIX command PROGRAM. Since these programs are called by compiled solvers, there is a substantial speed advantage over a design that writes the solver in a subroutine. The file MATRIX2.MAC contains the subroutines DUD and MARQ which illustrate the subroutine approach to nonlinear least squares and the power of the MATRIX command language. While these subroutines can be used, the NLLSQ and NL2SOL commands are substantially more powerful and orders of magnitide faster. The MATRIX command display routines OUTSTRING, OUTINTEGER and OUTDOUBLE can be used to "instrument" the solution process such that the user can see how the search is proceeding if B34S is running in the Display Manager. These commands will not work for batch jobs since the proper windows have not been opened unless the b34s2 procedure is used. The B34S Matrix Language design allows DO loops and IF structures to be in the command stream provided the commands are not given in manual mode. This is not possible with Speakeasy. The command call manual; can be placed anywhere in the job to place the processor in manual mode whereby commands could be entered or the process modified. The main purpose of manual mode is to allow the user to take control of how a subroutine is running. If call manual; is used in a MATRIX program under the Display Manager, the user is placed in the IMATRIX mode where the output to date can be seen, errors can be trapped and commands can be given to see what has been calculated to date and scripts can be submitted. If the user is debugging a subroutine it is often useful to be able to see what variables are active and possible modify the course of the subroutines execution in a real-time mode. The command call break('Are at position a'); can be used to trap execution in a loop. If any key has been hit prior to call break being found, the program will stop. Otherwise the command is ignored. This saves having the program checking for a key stroke as each command is being executed which will slow speed. If call break is executed, the user can then either allow execution to proceed or terminate the process then and there. If the MATRIX workspace is saved with the SAVE command, it can be restored after more B34S commands have run. Unlike most other B34S commands, in many cases the MATRIX command requires that commas be used. Unlike other B34S commands, errors are written both in the LOG file and at the exact command that has caused a problem. The command call echooff; can be used to reduce output when running a user SUBROUTINE, FUNCTION or PROGRAM. If there is some question on how a certain section is running, the command call echoon; can be used to echo commands as they are executed. In Matlab commands that end with ; are not echoed, while commands that do not end in ; are. This "design" requires the user to modify code during the debug phase. As a result errors can creep in. The echoon and echoff commands allow one to globally turn off output. Every attempt has been made to increase the speed of execution. Unlike Speakeasy, the CALL command is needed to execute PROGRAMS or SUBROUTINES. By this design, a great deal of search time is saved. To avoid I/O and to avoid possible unforseen conflicts over names, user SUBROUTINES, PROGRAMS or FUNCTIONS must be loaded to be found. Automatic loading of user commands slows execution substantially since the processor would have to look in all libraries to see if a built-in command had been replaced by a user subroutine, program or function. A major reason automatic loading was not implemented was to avoid the danger that the wrong subroutine could be loaded and a "wrong" calculation be made and not caught. A major objective of the MATRIX command is to give the user total control over the processing of data. Users can develope extensions and modifications of the statistics calculated by other B34S commands. Over the years, more and more of the regular commands will be converted to save data for later use in the matrix command. The library of applications SUBROUTINES, FUNCTIONS and PROGRAMS in c:\b34slm\matrix2.mac will be inhanced. Since these procedures are written in the B34S MATRIX language, they are self documenting. Facilities are provided whereby users can add to these libraries. Since all procedures have to be loaded, there is NO possibility that a user procedure can conflict with a built-in command unless it is explicitly loaded. The only conflict possible is with a MATRIX language command. This design prevents the MATLAB problem of suddenly key commands not working because the order of the library is changed due to a new toolkit being added. The routines in matrix2.mac are of two types. General purpose routines, such CFREQ which will calculate a cumulative frequency distribution, are documented BOTH in the TOOLKIT section which lists all routines and under the matrix command subroutine list. Commands such as this have test problems of the same name in matrix.mac. Other programs in matrix2.mac of less general interest for production jobs are only documented in the TOOLKIT section. The library staging.mac contains example files for the proposed subroutines in staging2.mac. The MATRIX command can read and process a binary file. Reading can be done in any order. This facility allows the system to process and change a load module or recover data from a file that has a strange structure. Character*1 processing allows the user to read and parse lines of a file. This capability is of interest to the expert user. For NLLSQ and maximum likelihood models that involve recursive models, in many cases an alternative program may be RATS. Such models are slow to estimate since there is a heavy parse overhead and the the vector capability in the B34S MATRIX facility is not usuable. B34S is not designed to replace RATS in this area. In B34S NLLSQ and maximum likelihood estimation can be done where the model is specified in the user's program using MATRIX commands. This mode of operation combines the advantages of fast execution with the flexibility of allowing the user to specify the model using a 4th generation language. NLLSQ can also be done using subroutines DUD and MARQ which were written in the B34S MATRIX command language. The estimation of a ML model when a recursive system is not required is fast and a number of routines from IMSL are supplied. Supported capability includes constrained ML models. While recursive systems are possible, due to the fact that DO loops or SOLVE/FORMULA commands are needed, the speed is slower. The subroutine GARCH can be used to estimate a subset of the ARCH/GARCH class of models without some of the recursive call overhead. An even simplier command GARCHEST allows univariate estimation for simple ARCH/GARCH models. Since B34S can estimate a nonlinear programming model with nonlinear constraints, a fairly wide class of models can be estimated. The BGARCH subroutine allows the user's PROGRAM to estimate a bivariate GARCH model without the recursive overhead. Note: In the above NLLSQ refers to nonlinear least squares. The B34S MATRIX command has two commands for this calculation. NLLSQ uses the Meeter (1964a, 1964b) routines that were first developed at the University of Wisconsin and form the basis of much work in time series analysis. The alternative routine NL2SOL uses the Dennis-Gay-Welsch (1981) programs that have been widely used in the literature. As of May 2004, the NLLSQ command can be run with real*8 or real*16 data. The maximization routines supported all come from various versions of the IMSL library. The B34S MATRIX command subroutines DUD and MARQ use the logic of the SAS Nonlin command but are written in the B34S command language. These are supplied for research interest. For nonlinear least squares, these would be a third choice. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ List of Built-In Matrix Command Subroutines ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ACEFIT - Alternating Conditional Expectation Model Estimation ACE_OLS - Select best ACE Model ACE_PLOT - ACE Plots ACF_PLOT - Simple ACF Plot ADDCOL - Add a column to a 2d array or matrix. ADDROW - Add a row to a 2d array or matrix. AGGDATA - Aggregate Data under control of an ID Vector. ALIGN - Align Series with Missing Data ARMA - ARMA estimation using ML and MOM. AUTOBJ - Automatic Estimation of Box-Jenkins Model BACKSPACE - Backspace a unit BDS - BDS Nonlinearity test. BESTREG - Best OLS REGRESSION B_G_TEST - Breusch-Godfrey (1978) Residual Test BGARCH - Calculate function for a BGARCH model. BLUS - BLUS Residual Analysis BPFILTER - Baxter-King Filter. BREAK - Set User Program Break Point. BUILDLAG - Builds NEWY and NEWX for VAR Modeling CCFTEST - Display CCF Function of Prewhitened data CHAR1 - Place a string is a character*1 array. CHARACTER - Place a string in a character*1 array. CHECKPOINT - Save workspace in portable file. CLEARALL - Clears all objects from workspace. CLEARDAT - Clears data from workspace. CLOSE - Close a logical unit. CLS - Clear screen. CLUSTER - K-Means and Hierarchical Cluster Models CMAXF1 - Constrained maximization of function using zxmwd. CMAXF2 - Constrained maximization of function using dbconf/g. CMAXF3 - Constrained maximization of function using db2pol. COMPRESS - Compress workspace. CONSTRAIN - Subset data based on range of values. CONTRACT - Contract a character array. COPY - Copy an object to another object COPYLOG - Copy file to log file. COPYOUT - Copy file to output file. COPYTIME - Copy time info from series 1 to series 2 COPYF - Copy a file from one unit to another. CPERIOD - Normalized Cumulative Periodogram CSPECTRAL - Do cross spectral analysis. CSUB - Call Subroutine CSV - Read and Write a CSV file DATA_ACF - Calculate ACF and PACF Plots DATA2ACF - Calculate ACF and PACF Plots added argument DATAFREQ - Data Frequency DATAVIEW - View a Series Under Menu Control DELETECOL - Delete a column from a matrix or array. DELETEROW - Delete a row from a matrix or array. DES - Code / decode. DESCRIBE - Calculate Moment 1-4 and 6 of a series DF - Calculate Dickey-Fuller Unit Root Test. DISPLAYB - Displays a Buffer contents DIST_TAB - Distribution Table DMFGET - Load data froma DMF Save file DMFPUT - Place data in a DMF save file DMFMERGE - Merge two DMF save files DODOS - Execute a command string if under dos/windows. DOT_PRODUCT - Use Fortran DOT_PRODUCT( ) and SUM( ) DOT_BLAS - USE BLAS DDOT and DSUM DO_SPEC - Display Periodogram and Spectrum DO2SPEC - Display Periodogram and Spectrum added argument DOUNIX - Execute a command string if under unix. DQDAG - Integrate a function using Gauss-Kronrod rules DQDNG - Integrate a smooth function using a nonadaptive rule. DQDAGI - Integrate a function over infinite/semi-infinite interval. DQDAGP - Integrate a function with singularity points given DQDAGS - Integrate a function with end point singularities DQAND - Multiple integration of a function DTWODQ - Two Dimensional Iterated Integral ESACF - Extended Sample Autocorrelation Function ECHOOFF - Turn off listing of execution. ECHOON - Turn on listing of execution. EPPRINT - Print to log and output file. EPRINT - Print to log file. ERASE - Erase file(s). EXPAND - Expand a character array FLS - Flexable Least Squares option FLSGRAPH - Graph FLS Results FLSFRONT - Graph FLS Model Frontier FORMS - Build Control Forms FORPLOT - Forecast Plot FREE - Free a variable. FPLOT - Plot a Function FPRINT - Formatted print facility. GAMFIT - Generalized Additive Model Estimation GARCH - Calculate function for a ARCH/GARCH model. GARCHEST - Estimate ARCH/GARCH model. GET - Gets a variable from b34s. GETDMF - Gets a data from a b34s DFM file. GETKEY - Gets a key GETMATLAB - Gets data from matlab. GET_FILE - Gets a File name GET_NAME - Get Name of a Matrix Variable GETRATS - Reads RATS Portable file. GETSCA - Reads SCA FSAVE and MAD portable files. GMFAC - LU factorization of n by m matrix GMINV - Inverse of General Matrix using LAPACK GMSOLV - Solve Linear Equations system using LAPACK GRAPH - High Resolution graph. GRAPHP - Multi-Pass Graphics Programing Capability GRCHARSET - Set Character Set for Graphics. GRREPLAY - Graph replay and reformat command. GTEST - Tests output of a ARCH/GARCH Model GWRITE - Save Objects in GAUSS Format using one file GWRITE2 - Save objects in GAUSS format using two files HEADER - Turn on header HEXTOCH - Concert hex to a character representation. HINICH82 - Hinich 1982 Nonlinearity Test. HINICH96 - Hinich 1996 Nonlinearity Test. HPFILTER - Hodrick-Prescott Filter. ISEXTRACT - Place data in a structure. IALEN - Get actual length of a buffer of character data IBFCLOSE - Close a file that was used for Binary I/O IBFOPEN - Open a File for Binary I/O IBFREADC - Reads from a binary file into Character*1 array IBFREADR - Reads from a binary file into Real*8 array IBFSEEK - Position Binary read/write pointer IBFWRITER - Write noncharacter buffer on a binary file IBFWRITEC - Write character buffer on a binary file IB34S11 - Parse a token using B34S11 parser IFILESIZE - Determine number of bites in a file IFILLSTR - Fill a string with a character IGETICHAR - Obtain ichar info on a character buffer IGETCHARI - Get character from ichar value IJUSTSTR - Left/Right/center a string ILCOPY - Move bites from one location to another ILOCATESTR - Locate a substring in a string - 200 length max ILOWER - Lower case a string - 200 length max INEXTR8 - Convert next value in string to real*8 variable INEXTR4 - Convert next value in string to real*4 variable INEXTSTR - Extract next blank deliminated sub-string from a string INEXTI4 - Convert next value in a string to integer. INTTOSTR - Convert integer to string using format IRF - Impulse Response Functions of VAR Model IR8TOSTR - Convert real*8 value to string using format ISTRTOR8 - Convert string to real*8 ISTRTOINT - Convert string to integer IUPPER - Upper case a string - 200 length max I_DRNSES - Initializes the table used by shuffled generators. I_DRNGES - Get the table used in the shuffled generators. I_DRNUN - Uniform (0,1) Generator I_DRNNOR - Random Normal Distribution I_DRNBET - Random numbers from beta distribution I_DRNCHI - Random numbers from Chi-squared distribution I_DRNCHY - Random numbers from Cauchy distribution I_DRNEXP - Random numbers from standard exponential I_DRNEXT - Random numbers from mixture of two exponential distributions I_DRNGAM - Random numbers from standard gamma distribution I_DRNGCT - Random numbers from general continuous distribution I_DRNGDA - Random integers from discrete distribution alias approach I_DRNGDT - Random integers from discrete using table lookup I_DRNLNL - Random numbers from lognormal distribution I_DRNMVN - Random numbers from multivariate normal I_DRNNOA - Random normal numbers using acceptance/rejection I_DRNNOR - Random normal numbers using CDF method I_DRNSTA - Random numbers from stable distribution I_DRNTRI - Random numbers from triangular distribution I_DRNSPH - Random numbers on the unit circle I_DRNVMS - Random numbers from Von Mises distribution I_DRNWIB - Random numbers from Weibull distribution I_RNBIN - Random integers from binomial distribution I_RNGET - Gets seed used in IMSL Random Number generators. I_RNOPG - Gets the type of generator currently in use. I_RNOPT - Selects the type of uniform (0,1) generator. I_RNSET - Sets seed used in IMSL Random Number generators. I_RNGEO - Random integers from Geometric distribution I_RNHYP - Random integers from Hypergeometric distribution. I_RNMTN - Random numbers from multinomial distribution I_RNNBN - Negative binomial distribution I_RNPER - Random perturbation of integers I_RNSRI - Index of random sample without replacement KEENAN - Keenan Nonlinearity test KPSS - KPSS Stationarity Test KSWTEST - K Period Stock Watson Test KSWBOOTS - Critical values for output from KSWTEST KSWTESTM - Moving Period Stock Watson Test LAGMATRIX - Builds Lag Matrix. LAGTEST - 3-D Graph to display RSS for OLS Lags LAGTEST2 - 3-D Graph to display RSS for MARS Lags LAPACK - Sets Key LAPACK parameters LM - Engle Lagrange Multiplier ARCH test. LOAD - Load a Subroutine from a library. LOADDATA - Load Data from b34s into MATRIX command. LPMAX - Solve Linear Programming maximization problem. LPMIN - Solve Linear Programming minimization problem. LRE - McCullough Log Relative Error LS2 - Two Stage Least Squares MAKEDATA - Place data in a b34s data loading structure. MAKEFAIR - Make Fair-Parke Data Loading File MAKEGLOBAL - Make a variable global (seen at all levels). MAKELOCAL - Make a variable seen at only local level. MAKEMATLAB - Place data in a file to be loaded into Matlab. MAKEMAD - Makes SCA *.MAD datafile from vectors MAKERATS - Make RATS portable file. MAKESCA - Make SCA FSV portable file. MANUAL - Place MATRIX command in manual mode. MARS - Multivariate Autoregressive Spline Models MARSPLINE - Updated MARS Command using Hastie-Tibshirani code MARS_VAR - Joint Estination of VAR Model using MARS Approach MAXF1 - Maximize a function using IMSL ZXMIN. MAXF2 - Maximize a function using IMSL DUMINF/DUMING. MAXF3 - Maximize a function using simplex method (DU2POL). MELD - Form all possible combinations of vectors. MENU - Put up user Menu for Input MESSAGE - Put up user message and allow a decision. MINIMAX - Estimate MINIMAX with MAXF2 MISSPLOT - Plot of a series with Missing Data MQSTAT - Multivariate Q Statistic MVNLTEST - Multivariate Third Order Hinich Test NAMES - List names in storage. NLEQ - Jointly solve a number of nonlinear equations. NLLSQ - Nonlinear Least Squares Estimation. NL2SOL - Alternative Nonlinear Least Squares Estimation. NLPMIN1 - Nonlinear Programming fin. diff. grad. DN2CONF. NLPMIN2 - Nonlinear Programming user supplied grad. DN2CONG. NLPMIN3 - Nonlinear Programming user supplied grad. DN0ONF. NLSTART - Generate starting values for NL routines. NOHEADER - Turn off header. OLSQ - Estimate OLS, MINIMAX and L1 models. OLSPLOT - Plot of Fitted and Actual Data & Res OPEN - Open a file and attach to a unit. OUTDOUBLE - Display a Real*8 value at a x, y on screen. OUTINTEGER - Display an Integer*4 value at a x, y on screen. OUTSTRING - Display a string value at a x, y point on screen. PCOPY - Copy an object from one pointer address to another PERMUTE - Reorder Square Matrix PISPLINE - Pi Spline Nonlinear Model Building PLOT - Line-Printer Graphics POLYFIT - Fit an nth degree polynomial POLYVAL - Evaluate an nth degree polynomial POLYMCONV - Convert storage of a polynomial matrix POLYMDISP - Display/Extract a polynomial matrix POLYMINV - Invert a Polynomial Matrix POLYMMULT - Multiply a Polynomial Matrix PPEXP - Two-dimensional exploratory projection pursuit PPEXP_P - Plot and save ppexp output PPREG - Projection Pursuit Regression PP - Calculate Phillips Peron Unit Root test PRINT - Print text and data objects. PRINTALL - Lists all variables in storage. PRINTOFF - Turn off Printing PRINTON - Turn on Printing (This is the default) PRINTVASV - Resets so that vectors/arrays print as vectors/arrays PRINTVASCMAT - Vectors/Arrays print as Column Matrix/Array PRINTVASRMAT - Vectors/Arrays print as Row Matrix/Array PROBIT - Estimate Probit (0-1) Model. PVALUE_1 - Present value of $1 recieved at end of n years PVALUE_2 - Present value of an Annuity of $1 PVALUE_3 - Present value of $1 recieved throughout year QPMIN - Quadratic Programming. QUANTILE - Calculate interquartile range. RANFOREST - Call the Random Forest Capability RCOVER - Recursive Covering DART/HYESS READ - Read data directly into MATRIX workspace from a file. REAL16INFO - Obtain Real16 info REAL16OFF - Turn off Real16 add REAL16ON - Turn on extended accuracy REAL32OFF - Turn off Real32 add REAL32ON - Turn on extended accuracy for real*16 REAL32_VPA - Turn on extended accuracy for real*16 using vpa RESET - Calculate Ramsey (1969) regression specification test. RESET77 - Thursby - Schmidt Regression Specification Test RESTORE - Load data back in MATRIX facility from external save file. RTEST - Test Residuals of Model RTEST2 - Test Residuals of Model - No RES and Y Plots REVERSE - Test a real*8 vector for reversibility in Freq. Domain REWIND - Rewind logical unit. ROTHMAN - Test a real*8 vector for reversibility in Time Domain RMATLAB - Runs Matlab RRPLOTS - Plots Recursive Residual Data RRPLOTS2 - Plots Recursive Residual Coef RUN - Terminates the matrix command being in "manual" mode. SAVE - Save current workspace in portable file format. SC_TEST - Serial Correlation Tests SCHUR - Performs Schur decomposition SCREENCLOSE - Turn off Display Manager SCREENOPEN - Turn on Display Manager SCREENOUTOFF - Turn screen output off. SCREENOUTON - Turn screen output on. SET - Set all elements of an object to a value. SETCOL - Set column of an object to a value. SETLABEL - Set the label of an object. SETLEVEL - Set level. SETNDIMV - Sets an element in an n dimensional object. SETROW - Set row of an object to a value. SETTIME - Sets the time info in an existing series SETWINDOW - Set window to main(1), help(2) or error(3). SIGD - Set print digits. Default g16.8 SIMULATE - Dynamically Simulate OLS Model SMOOTH - Do exponential smoothing. SOLVEFREE - Set frequency of freeing temp variables. SORT - Sort a real vector. SPECTRAL - Spectral analysis of a vector or 1d array. STEPWISE - Stepwise OLS Regression STOP - Stop execution of a matrix command. SUBRENAME - Internally rename a subroutine. SUSPEND - Suspend loading and Execuiting a program SWARTEST - Stock-Watson VAR Test SYSTEM - Issue a system command. TABULATE - List vectors in a table. TB_TO_JULIAN - Converts tbase tstart freq to a real*8 julian date TESTARG - Lists what is passed to a subroutine or function. TIMER - Gets CPU time. TRIPLES - Calculate Triples Reversability Test TSAY - Calculate Tsay nonlinearity test. TSLINEUP - Line up Time Series Data TSD - Interface to TSD Data set UP_DATE_TB - Update the object header info UROOT_T - Battery of DF, PP, ERS and KPSS test VAREST - VAR Modeling VPASET - Set Variable Precision Math Options VOCAB - List built-in subroutine vocabulary. WRITE - Write an object to an external file. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Matrix Command Built-In Function Vocabulary +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ACF - Calculate autocorrelation function of a 1d object. AFAM - Change a matrix or vector to an array class object. ARGUMENT - Unpack character argument at run-time ARRAY - Define a 1d or 2d array. BETAPROB - Calculate a beta probability. BINDF - Evaluate Binomial Distribution Function BINPR - Evaluate Binomial Probability Function BOOTI - Calculate integers to be used with bootstrap. BOOTV - Bootstraps a vector with replacement. BOXCOX - Box-Cox Transformation of a series given lamda. BSNAK - Compute Not a Knot Sequence BSOPK - Compute optimal spline knot sequence BSINT - Compute 1-D spline interpolant given knots BSINT2 - Compute 2-D spline interpolant given knots BSINT3 - Compute 3-D spline interpolant given knots BSDER - Compute 1-D spline values/derivatives given knots BSDER2 - Compute 2-D spline values/derivatives given knots BSDER3 - Compute 3-D spline values/derivatives given knots BSITG - Compute 1-D spline integral given knots BSITG2 - Compute 2-D spline integral given knots BSITG3 - Compute 3-D spline integral given knots C1ARRAY - Create a Character*1 array C8ARRAY - Create a Character*8 array CATCOL - Concatenates an object by columns. CATROW - Concatenates an object by rows. CCF - Calculate the cross correlation function on two objects. CHAR - Convect an integer in range 0-127 to character. CHARDATE - Convert julian variable into character date dd\mm\yy. CHARDATEMY - Convert julian variable into character data mm\yyyy. CHARTIME - Converts julian variable into character date hh:mm:ss CHISQPROB - Calculate chi-square probability. CHTOR - Convert a character variable to a real variable. CHTOHEX - Convert a character to its hex representation. CFUNC - Call Function COMB - Combination of N objects taken M at a time. COMPLEX - Build a complex variable from two real*8 variables. CSPLINEFIT - Fit a 1 D Cubic Spline using alternative models CSPLINE - Calculate a cubic spline for 1 D data CSPLINEVAL - Calculate spline value given spline CSPLINEDER - Calculate spline derivative given spline value CSPLINEITG - Calculate integral of a cubic spline CUSUM - Cumulative sum. CUSUMSQ - Cumulative sum squared. CWEEK - Name of the day in character. DABS - Absolute value of a real*8 variable. DARCOS - Arc cosine of a real*8 variable. DARSIN - Arc sine of a real*8 variable. DATAN - Arc tan of a real*8 variable. DATAN2 - Arc tan of x / y. Signs inspected. DATENOW - Date now in form dd/mm/yy DBLE - Convert real*4 to real*8. DCONJ - Conjugate of complex argument. DCOS - Cosine of real*8 argument. DCOSH - Hyperbolic cosine of real*8 argument. DDOT - Inner product to two vectors. DERF - Error function of real*8/real*16 argument. DERFC - Inverse of error function. DERIVATIVE - Analytic derivative of a vector. DET - Determinate of a matrix. DEXP - Exponential of a real*8 argument. DFLOAT - Convert integer*4 to real*8. DGAMMA - Gamma function of real*8 argument. DIAG - Place diagonal of a matrix in an array. DIAGMAT - Create diagonal matrix. DIF - Difference a series. DINT - Extract integer part of real*8 number DNINT - Extract nearest integer part of real*8 number DIVIDE - Divide with an alternative return. DLGAMMA - Natural log of gamma function. DLOG - Natural log. DLOG10 - Base 10 log. DMAX - Largest element in an array. DMAX1 - Largest element between two arrays. DMIN - Smallest element in an array. DMIN1 - Smallest element between two arrays. DMOD - Remainder. DROPFIRST - Drops observations on top or array. DROPLAST - Drops observations on bottom of an array. DSIN - Calculates sine. DSINH - Hyperbolic sine. DSQRT - Square root of real*8 or complex*16 variable. DTAN - Tangent. DTANH - Hyperbolic tangent. EIGENVAL - Eigenvalue of matrix. Alias EIG. EPSILON - Positive value such that 1.+x ne 1. EVAL - Evaluate a character argument EXP - Exponential of real*8 or complex*16 variable. EXTRACT - Extract elements of a character*1 variable. FACT - Factorial FDAYHMS - Gets fraction of a day. FFT - Fast fourier transform. FIND - Finds location of a character string. FLOAT - Converts integer*4 to real*4. FPROB - Probability of F distribution. FREQ - Gets frequency of a time series. FRACDIF - Fractional Differencing FYEAR - Gets fraction of a year from julian date. GENARMA - Generate an ARMA series given parameters. GETDAY - Obtain day of year from julian series. GETHOUR - Obtains hour of the day from julian date. GETNDIMV - Obtain value from an n dimensional object. GETMINUTE - Obtains minute of the day from julian date. GETMONTH - Obtains month from julian date. GETQT - Obtains quarter of year from julian date. GETSECOND - Obtains second from julian date. GETYEAR - Obtains year. GOODCOL - Deletes all columns where there is missing data. GOODROW - Deletes all rows where there is missing data. GRID - Defines a real*8 array with a given increment. HUGE - Largest number of type HYPDF - Evaluate Hypergeometric Distribution Function HYPPR - Evaluate Hypergeometric Probability Function HC_SIGMA - Het-Consistent estimate of Sigma INTEGER8 - Load an Integer*8 object from a string I4TOI8 - Move an object from integer*4 to integer*8 I8TOI4 - Move an object from integer*8 to integer*4 ICHAR - Convect a character to integer in range 0-127. ICOLOR - Sets Color numbers. Used with Graphp. IDINT - Converts from real*8 to integer*4. IDNINT - Converts from real*8 to integer*4 with rounding. INFOGRAPH - Obtain Interacter Graphics INFO IMAG - Copy imaginary part of complex*16 number into real*8. IAMAX - Largest abs element in 1 or 2D object IAMIN - Smallest abs element in 1 or 2D object IMAX - Largest element in 1 or 2D object IMIN - Smallest element in 1 or 2D object INDEX - Define integer index vector, address n dimensional object. INLINE - Inline creation of a program INT - Copy real*4 to integer*4. INTEGERS - Generate an integer vector with given interval. INV - Inverse of a real*8 or complex*16 matrix. INVBETA - Inverse beta distribution. INVCHISQ - Inverse Chi-square distribution. INVFDIS - Inverse F distribution. INVTDIS - Inverse t distribution. IQINT - Converts from real*16 to integer*4. IQNINT - Converts from real*16 to integer*4 with rounding. ISMISSING - Sets to 1.0 if variable is missing IWEEK - Sets 1. for monday etc. JULDAYDMY - Given day, month, year gets julian value. JULDAYQY - Given quarter and year gets julian value. JULDAYY - Given year gets julian value. KEEPFIRST - Given k, keeps first k observations. KEEPLAST - Given k, keeps last k observations. KIND - Returns kind of an object in integer. KINDAS - Sets kind of second argument to kind first arg. KLASS - Returns klass of an object in integer. KPROD - Kronecker Product of two matrices. LABEL - Returns label of a variable. LAG - Lags variable. Missing values propagated. LEVEL - Returns current level. LOWERT - Lower triangle of matrix. MCOV - Consistent Covariance Matrix MAKEJUL - Make a Julian date from a time series MASKADD - Add if mask is set. MASKSUB - Subtract if mask is set. MATRIX - Define a matrix. MEAN - Average of a 1d object. MEDIAN - Median of a real*8 object MFAM - Set 1d or 2d array to vector or matrix. MISSING - Returns missing value. MLSUM - Sums log of elements of a 1d object. MOVELEFT - Moves elements of character variable left. MOVERIGHT - Move elements of character variable right. NAMELIST - Creates a namelist. NEAREST Nearest distinct number of a given type NCCHISQ - Non central chi-square probability. NOCOLS - Gets number of columns of an object. NOELS - Gets number of elements in an object. NORMDEN - Normal density. NORMDIST - 1-norm, 2-norm and i-norm distance. NOROWS - Gets number of rows of an object. NOTFIND - Location where a character is not found. OBJECT - Put together character objects. PDFAC - Cholesky factorization of PD matrix. PDFACDD - Downdate Cholesky factorization. PDFACUD - Update Cholesky factorization. PDINV - Inverse of a PD matrix. PDSOLV - Solution of a PD matrix given right hand side. PI - Pi value. PINV - Generalized inverse. PLACE - Places characters inside a character array. POIDF - Evaluate Poisson Distribution Function POIPR - Evaluate Poisson Probability Function POINTER - Machine address of a variable. POLYDV - Division of polynomials. POLYMULT - Multiply two polynomials POLYROOT - Solution of a polynomial. PROBIT - Inverse normal distribution. PROBNORM - Probability of normal distribution. PROBNORM2 - Bivariate probability of Normal distribution. PROD - Product of elements of a vector. Q1 - Q1 of a real*8 object Q3 - Q3 of a real*8 object QCOMPLEX - Build complex*32 variable from real*16 inputs. QINT - Extract integer part of real*16 number QNINT - Extract nearest integer part of real*16 number QREAL - Obtain real*16 part of a complex*326 number. QR - Obtain Cholesky R via QR method using LAPACK. QRFAC - Obtain Cholesky R via QR method. QRSOLVE - Solve OLS using QR. RANK - Rank of a matrix RANKER - Index array that ranks a vector. RCOND - 1 / Condition of a Matrix REAL - Obtain real*8 part of a complex*16 number. R8TOR16 - Convert Real*8 to Real*16 R16TOR8 - Convert Real*16 to Real*8 REAL16 - Input a Real*16 Variable REC - Rectangular random number. RECODE - Recode a real*8 or character*8 variable RN - Normally distributed random number. ROLLDOWN - Moves rows of a 2d object down. ROLLLEFT - Moves cols of a 2d object left. ROLLRIGHT - Moves cols of a 2d object right. ROLLUP - Moves rows of a 2d object up. RTOCH - Copies a real*8 variable into character*8. SEIGENVAL - Eigenvalues of a symmetric matrix. Alias SEIG. SEXTRACT - Takes data out of a field. SFAM - Creates a scalar object. SNGL - Converts real*8 to real*4. SPACING - Absolute spacing near a given number SPECTRUM - Returns spectrum of a 1d object. SUBSET - Subset 1d, 2d array, vector or matrix under a mask. SUBMATRIX - Define a Submatrix SUM - Sum of elements. SUMCOLS - Sum of columns of an object. SUMROWS - Sum of rows of an object. SUMSQ - Sum of squared elements of an object. SVD - Singular value decomposition of an object. TIMEBASE - Obtains time base of an object. TIMENOW - Time now in form hh:mm:ss TIMESTART - Obtains time start of an object. TINY - Smallest number of type TDEN - t distribution density. TO_RMATRIX - Convert Object to Row-Matrix TO_CMATRIX - Convert Object to Col-Matrix TO_RARRAY - Convert Object to Row-Array TO_CARRAY - Convert Object to Col-Matrix TO_VECTOR - Convert Object to Vector TO_ARRAY - Convert Object to Array TPROB - t distribution probability. TRACE - Trace of a matrix. TRANSPOSE - Transpose of a matrix. UPPERT - Upper Triangle of matrix. VARIANCE - Variance of an object. VECTOR - Create a vector. VFAM - Convert a 1d array to a vector. VOCAB - List built in functions. VPA - Variable Precision Math calculation ZDOTC - Conjugate product of two complex*16 objects. ZDOTU - Product of two complex*16 objects. ZEROL - Zero lower triangle. ZEROU - Zero upper triangle. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Matrix Programming Language key words ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ CALL - Call a subroutine CONTINUE - go to statement DO - Starts a do loop DOWHILE - Starts a dowhile loop NEXT i - End of a do loop ENDDO - End of a do loop ENDDOWHILE - End of a dowhile loop END - End of a program, function or Subroutine. EXITDO - Exit a DO loop EXITIF - Exit an IF statement FOR - Start a do loop, FORMULA - Define a recursive formula. GO TO - Transfer statement FUNCTION - Beginning of a function. IF( ) - Beginning of an IF structure ENDIF - End of an IF( )THEN structure PROGRAM - Beginning of a program, RETURN - Next to last statement before end. RETURN( ) - Returns the result of a function. SOLVE - Solve a recursive system. SUBROUTINE - Beginning of subroutine. WHERE( ) - Starts a where structure. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ matrix2.mac capability as of 1 May 2007 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Mac file name c:\b34slm\lib\matrix2.mac Name Heading ACF_PLOT Plot ACF ALTSE Copies lower triangle to upper and gets SE AUTOCOV Autocovaiance BJ_IDEN List / Plot ACF and PACF B_G_TEST Breusch- Godfrey (1978) Test on Residuals B_G_ALT Breusch- Godfrey (1978) Test using dropping BLUS BLUS master BPF Baxter - King Symmetric MA Filter BPFM Baxter - King Symmetric MA Filter - Missing Data at BUILDLAG Builds NEWY and NEWX for VAR Modeling CANCORR Calculate Canonical Correlations CATS Cointegration Test using Johansen Method CCFTEST Tests Cross Correlations CFREQ Determine Frequency Distribution COINT2 Cointegreation of Two Series COINT2LM Cointegreation of Two Series - Gives L1 and Minimax COINT2M Moving Cointegration of Two Series - Simple Version COINT2M2 Moving Cointegration Two Series OLS, L1 Minimax COINT2ME Moving Cointegration of Two Series - Extended Versi COINT3 Cointegration of Three Series COINT3ME Moving Cointegration of Three Series COINT_SW Stock-Watson DOLS Analysis DATAVIEW Interactively View Data DATA_ACF Plot ACF and PACF of a series DATA2ACF Plot ACF and PACF of a series with file name DF_GLS Elliot-Rothenberg-Stock DF_GLS Test DIST_TAB Distribution Table DO_SPEC View Spectrum DO2SPEC View Spectrum with a file name DUD Derivative Free Nonlinear Estimation DUD2 Derivative Free Nonlinear Estimation FDIFINFO Fractional Differencing FILTER High Low Pass Filter using Real FFT FILTERC High Low Pass Filter using Complex FFT FORPLOT Forecast Plot GAMFORE Forecast a GAM Model GAMPLOT Display GAMFIT Results GARCH2P Two Pass GARCH Using ARMA Command GARCH2PF Two Pass GARCH Using ARMA Command with Forecasts GET_NAME Get Name of a Matrix Variable GTEST Tests ARCH / GARCH Models GWRITE Save Matrix Object in GAUSS Format using one file GWRITE2 Pass Large Datasets to GAUSS in 2 files HP_2 Hodrick - Prescott Moving Filtering HP_BP_1 Baxter - King & Hodrick - Prescott Filtering HP_BP_2 Baxter - King & Hodrick - Prescott Moving Filtering IRF Impulse Response Functions for VAR Model KPSS KPSS Stationarity Test KSWTEST K Period Stock Watson Test KSWBOOTS Critical values for output from KSWTEST KSWTESTM Moving Period Stock Watson Test LAGTEST 3-D Graph to display RSS for Various OLS Lags LAGTEST2 3-D Graph to display RSS for Various MARS Lags LMTEST Engle LM test for a Range of lags MARQ Estimation of a Nonlinear Model using Derivatives MARSPLOT Plot MARS Curves & Surfaces MCLEODLI McLeod-Li (1983) Linearity test (y,ip,maxacf) MINIMAX Minimax Estimation using MAXF2 MISSPLOT Plots Data With Missing Values MOVEAVE Moving average of a vector MOVEBJ Moving Arima Forecast using AUTOBJ MOVECORR Moving Correlation of two vectors MOVEH82 Moving Hinich 82 test MOVEH96 Moving Hinich 96 test MOVEOLS Moving OLS with LAGS MOVEVAR Moving Variance NLVARCOV NLLSQ Variance Covariance OLSPLOT Plot of Fitted and Actual Data & Res PAD Pad a 1D Real*8 Series on both ends PERMUTE Reorder a Square Matrix POLYFIT Fit an nth Order Regression POLYVAL Evaluate an nth Order Polynominam Regression PVALUE_1 Present value of $1 recieved at end of n years PVALUE_2 Present Value of an Annuity of $1 PVALUE_3 Present value of $1 recieved throughout year QUANTREG Quantile Regression RESET69 Ramsey(1969) Regression Specification Test RESET77 RESET77 Nonlinearitry Test RMATLAB Runs Matlab commands RRPLOTS Plots Recursive Residual Data RRPLOTS2 Plots Recursive Coef and t scores RTEST Tests Residuals of TS Models RTEST2 Tests Residuals of TS Models - No Res and y Plots SUBSET Subset an array under mask control SWARTEST Stock-Watson AR Test TESTFUN Test Function TESTPGM Test Program TESTSUB Test Subroutine UROOT_T Battery of DF, PP, ERS and KPSS test VAREST VAR Modeling ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ staging2.mac capability as of 1 May 2007 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Mac file name c:\b34slm\lib\staging2.mac Name Heading ACE_OLS Tests ACE Model using OLS ACE_PLOT Plot ACE Response Curves BOOTOLS Bootstrap OLS Model BOOTL1 Bootstrap L1 Model BOOTMM Bootstrap MM Model COR Correlation of a matrix. COV Calculated Covariance of a matrix COND Condition of a Matrix DFVALUES Estimate of DF/PP Critical Values DISPMARS Displays Mars Model F_MARSLG Adjusts yhat and forecasts from MARSPLINE G4TEST Pena-Slate JASA March (2006) Joint OLS Test GAMPLOT Display GAMFIT Results GARCHF Forecast ARCH / GARCH Series GARCH2PA Automatic GARCH Two Pass Modeling GENGARCH Simulate GARCH and ARCH Models GETCHAR Unpack a string into Character Array GETDATA Simple Real*8 data Read routine GETDATA2 Simple Real*16 data Read routine GETR16 Unpack a string into Real*16 Array GETVPA Unpack a string into VPA Array GETR8 Unpack a string into Real*8 Array GLESJER Glesjer (1969) Heteroscedasticity Test GLS Cochrane & Orcutt GLS Estimation of AR(k) Model GLSDATA Transform x to x* given GLS rho vector GLS_ML GLS_ML Maximum Likelihood estimation of AR(1) Model GLS_2P Two Pass Generalized Least Squares for Order K GPH Geweke & Porter-Hudak (1983) Fractional Diff. Test G_QUANDT Goldfeld-Quandt Heteroscedasticity Test HET_TEST Heteroscedasticity Testing IACF Inverse ACF LASSO LASSO Shrinkage Approach MARSINFO Estimate of MARSPLINE Model GCV Info MCOVF Function for Covariance. See built in mcov( ) NORM NORM(X,i) of a matrix or vector NTOKIN Counts number of tokins in a string NW_SE Newey West SE Correction Calculation OLS_CON Constrained OLS Model P_L_EST Probit & Logit Estimation PANEL_LIB Panel Subroutine Library QR_SMALL Break Appart QR Factorization RIDGE Ridge Regression RNW_SE Run Newey West SE Correction for OLS Model SC_TEST Serial Correlation Tests SVD_OLS OLS Using SVD Approach SVD2_OLS OLS Using SVD Approach and LAPACK TLOGIT Tests Logit/Probit Model TS_TESTS Residual Specification tests TSALIGN Align Time Series Data UNDIF Undifference a series WALD Wald Test of restrictions +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Commands listed by Function Character & String Subroutines CHTOHEX - Convert a character to its hex representation. CONTRACT - Contract a character array DISPLAYB - Displays a Buffer contents EXPAND - Expand a character array EVAL - Evaluate a Character Argument HEXTOCH - Concert hex to a character representation. IALEN - Get actual length of a buffer of character data IBFCLOSE - Close a file that was used for Binary I/O IBFOPEN - Open a File for Binary I/O IBFREADC - Reads from a binary file into Character*1 array IBFREADR - Reads from a binary file into Real*8 array IBFSEEK - Position Binary read/write pointer IBFWRITER - Write noncharacter buffer on a binary file IBFWRITEC - Write character buffer on a binary file IB34S11 - Parse a token using B34S11 parser IFILESIZE - Determine number of bites in a file IFILLSTR - Fill a string with a character IGETICHAR - Obtain ichar info on a character buffer IGETCHARI - Get character from ichar value IJUSTSTR - Left/Right/center a string ILCOPY - Move bites from one location to another ILOCATESTR - Locate a substring in a string - 200 length max ILOWER - Lower case a string - 200 length max INEXTR8 - Convert next value in string to real*8 variable INEXTR4 - Convert next value in string to real*4 variable INEXTSTR - Extract next blank deliminated sub-string from a string INEXTI4 - Convert next value in a string to integer. INTTOSTR - Convert integer to string using format IR8TOSTR - Convert real*8 value to string using format ISTRTOR8 - Convert string to real*8 ISTRTOINT - Convert string to integer IUPPER - Upper case a string - 200 length max Character Functions CHAR - Convect an integer in range 0-127 to character. CHTOR - Convert a character variable to a real variable. DATENOW - Date now in form dd/mm/yy EXTRACT - Extract elements of a character*1 variable. FIND - Finds location of a character string. ICHAR - Convect a character to integer in range 0-127. INLINE - Inline creation of a program MOVELEFT - Moves elements of character variable left. MOVERIGHT - Move elements of character variable right. NAMELIST - Creates a namelist. NOTFIND - Location where a character is not found. OBJECT - Put together character objects. PLACE - Places characters inside a character array. RTOCH - Copies a real*8 variable into character*8. TIMENOW - Time now in form hh:mm:ss Data Building, loading and Display Subroutines ACF_PLOT - Simple ACF Plot ADDCOL - Add a column to a 2d array or matrix. ADDROW - Add a row to a 2d array or matrix. AGGDATA - Aggregate Data under control of an ID Vector. ALIGN Align Series with Missing Data BJ_IDEN - ACF and PACF Listing and or Plotting BUILDLAG - Builds NEWY and NEWX for VAR Modeling CCFTEST - Display CCF Function of Prewhitened data CHAR1 - Place a string in a character*1 array. CHARACTER - Place a string in a character*1 array. CHECKPOINT - Save workspace in portable file. CLEARALL - Clears all objects from workspace. CLEARDAT - Clears data from workspace. CSUB - Call Subroutine CONSTRAIN - Subset data based on range of values. CSV - Read and Write a CSV file DATA_ACF - Calculate ACF and PACF Plots DATA2ACF - Calculate ACF and PACF Plots added argument DATAVIEW - View a Series Under Menu Control DATAFREQ - Data Frequency DELETECOL - Delete a column from a matrix or array. DELETEROW - Delete a row from a matrix or array. DES - Code / decode. DESCRIBE - Calculate Moment 1-4 and 6 of a series DIVIDE - Divide with an alternative return. DO_SPEC - Display Periodogram and Spectrum DO2SPEC - Display Periodogram and Spectrum added argument ESACF - Extended Sample Autocorrelation Function GET_FILE - Get a File Name GET_NAME - Get Name of a Matrix Variable GTEST - Tests output of a ARCH/GARCH Model GWRITE - Save Objects in GAUSS Format using one file GWRITE2 - Save Objects in GAUSS Format using two files FREE - Free a variable. LAGMATRIX - Builds Lag matrix. MELD - Form all possible combinations of vectors. QUANTILE - Calculate interquartile range. RTEST - Test Residuals of Model RTEST2 - Test Residuals of OLS Model - No RES and Y Plots SET - Set all elements of an object to a value. SETCOL - Set column of an object to a value. SETLABEL - Set the label of an object. SETNDIMV - Sets an element in an n dimensional object. SETROW - Set row of an object to a value. SETTIME - Sets the time info in an existing series SORT - Sort a real vector. SUBRENAME - Internally rename a subroutine. Data Building Functions AFAM - Change a matrix or vector to an array class object. ARRAY - Define a 1d or 2d array. C1ARRAY - Create a Character*1 array C8ARRAY - Create a Character*8 array CATCOL - Concatenates an object by columns. CATROW - Concatenates an object by rows. CFUNC - Call Function COMB - Combination of N objects taken M at a time. COMPLEX - Build a complex variable from two real*8 variables. CUSUM - Cumulative sum. CUSUMSQ - Cumulative sum squared. DABS - Absolute value of a real*8 variable. DARCOS - Arc cosine of a real*8 variable. DARSIN - Arc sine of a real*8 variable. DATAN - Arc tan of a real*8 variable. DATAN2 - Arc tan of x / y. Signs inspected. DBLE - Convert real*4 to real*8. DCONJ - Conjugate of complex argument. DCOS - Cosine of real*8 argument. DCOSH - Hyperbolic cosine of real*8 argument. DDOT - Inner product to two vectors. DEXP - Exponential of a real*8 argument. DFLOAT - Convert integer*4 to real*8. DGAMMA - Gamma function of real*8 argument. DINT - Extract integer part of real*8 number DNINT - Extract integer part of real*8 number DLGAMMA - Natural log of gamma function. DLOG - Natural log. DLOG10 - Base 10 log. DMAX - Largest element in an array. DMAX1 - Largest element between two arrays. DMIN - Smallest element in an array. DMIN1 - Smallest element between two arrays. DMOD - Remainder. DROPFIRST - Drops observations on top or array. DROPLAST - Drops observations on bottom of an array. DSIN - Calculates sine. DSINH - Hyperbolic sine. DSQRT - Square root of real*8 or complex*16 variable. DTAN - Tangent. DTANH - Hyperbolic tangent. EPSILON - Positive value such that 1.+x ne 1. EXP - Exponential of real*8 or complex*16 variable. FLOAT - Converts integer*4 to real*4. FREQ - Gets frequency of a time series. FYEAR - Gets fraction of a year from julian date. GETIDIM - Obtain value from an n dimensional object. GMFAC - LU factorization of n by m matrix GMINV - Inverse of General Matrix using LAPACK GMSOLV - Solve Linear Equations system using LAPACK GOODCOL - Deletes all columns where there is missing data. GOODROW - Deletes all rows where there is missing data. GRID - Defines a real*8 array with a given increment. HUGE - Largest number of type IAMAX - Largest abs element in 1 or 2D object IAMIN - Smallest abs element in 1 or 2D object IMAX - Largest element in 1 or 2D object IMIN - Smallest element in 1 or 2D object INTEGER8 - Load an Integer*8 object from a string I4TOI8 - Move an object from integer*4 to integer*8 I8TOI4 - Move an object from integer*8 to integer*4 ICOLOR - Sets Color numbers. Used with Graphp. IDINT - Converts from real*8 to integer*4. IDNINT - Converts from real*8 to integer*4 with rounding. IMAG - Copy imaginary part of complex*16 number into real*8. INDEX - Define integer index vector. INFOGRAPH - Obtain Interacter Graphics INFO INT - Copy real*4 to integer*4. INTEGERS - Generate an integer vector with given interval. IQINT - Converts from real*16 to integer*4. IQNINT - Converts from real*16 to integer*4 with rounding. ISMISSING - Sets to 1.0 if variable is missing KEEPFIRST - Given k, keeps first k observations. KEEPLAST - Given k, keeps last k observations. KIND - Returns kind of an object in integer. KINDAS - Sets kind of second argument to kind first arg. KLASS - Returns klass of an object in integer. LABEL - Returns label of a variable. LAG - Lags variable. Missing values propagated. LOWERT - Lower triangle of matrix. MCOV - Consistent Covariance Matrix MASKADD - Add if mask is set. MASKSUB - Subtract if mask is set. MATRIX - Define a matrix. MFAM - Set 1d or 2d array to vector or matrix. MISSING - Returns missing value. NEAREST Nearest distinct number of a given type NOCOLS - Gets number of columns of an object. NOELS - Gets number of elements in an object. NOROWS - Gets number of rows of an object. NORMDIST - 1-norm, 2-norm and i-norm distance. PI - Pi value. QCOMPLEX - Build complex*32 variable from real*16 inputs. QINT - Extract integer part of real*16 number QNINT - Extract nearest integer part of real*16 number QREAL - Obtain real*16 part of a complex*32 number. RANK - Rank of a matrix RANKER - Index array that ranks a vector. REAL - Obtain real*8 part of a complex*16 number. R8TOR16 - Convert Real*8 to Real*16 R16TOR8 - Convert Real*16 to Real*8 REAL16 - Input a Real*16 Variable REC - Rectangular random number. RECODE - Recode a real*8 or chartacter*8 variable RN - Normally distributed random number. SEXTRACT - Takes data out of a field. SFAM - Creates a scalar object. SNGL - Converts real*8 to real*4. SPACING Absolute spacing near a given number SUBMATRIX - Define a Submatrix SUBSET - Subset 1d, 2d array, vector or matrix under a mask. SUM - Sum of elements. SUMCOLS - Sum of columns of an object. SUMROWS - Sum of rows of an object. SUMSQ - Sum of squared elements of an object. TIMEBASE - Obtains time base of an object. TIMESTART - Obtains time start of an object. TINY Smallest number of type TO_RMATRIX - Convert Object to Row-Matrix TO_CMATRIX - Convert Object to Col-Matrix TO_RARRAY - Convert Object to Row-Array TO_CARRAY - Convert Object to Col-Matrix TO_VECTOR - Convert Object to Vector TO_ARRAY - Convert Object to Array UPPERT - Upper Triangle of matrix. VECTOR - Create a vector. VFAM - Convert a 1d array to a vector. ZDOTC - Conjugate product of two complex*16 objects. ZDOTU - Product of two complex*16 objects. ZEROL - Zero lower triangle. ZEROU - Zero upper triangle. Data Filtering Subroutines CSPECTRAL - Do cross spectral analysis. SPECTRAL - Spectral analysis of a vector or 1d array. LOESS_MV - Multivariate Loess Smoothing LOESS_SA - Seasonal Adjustment using LOESS Method. LOESS_SPS - Scatter Plot Smoothing using LOESS Method. FFT - Fast fourier transform. PPEXP - Two-dimensional exploratory projection pursuit PPEXP_P - Plot and save ppexp output PPREG - Projection Pursuit Regression RANFOREST - Call the Random Forest Capability RCOVER - Recursive Covering DART/HYESS SPECTRUM - Returns spectrum of a 1d object. VAREST - VAR Modeling WAVELET - Calculate Wavelets Date and Time Functions CHARDATE - Convert julian variable into character date dd\mm\yy. CHARDATEMY - Convert julian variable into character data mm\yyyy. CHARTIME - Converts julian variable into character date hh:mm:ss COPYTIME - Copy time info from series 1 to series 2 CWEEK - Name of the day in character. FDAYHMS - Gets fraction of a day. GETDAY - Obtain day of year from julian series. GETHOUR - Obtains hour of the day from julian date. GETMINUTE - Obtains minute of the day from julian date. GETMONTH - Obtains month from julian date. GETQT - Obtains quarter of year from julian date. GETSECOND - Obtains second from julian date. GETYEAR - Obtains year. IWEEK - Sets 1. for monday etc. JULDAYDMY - Given day, month, year gets julian value. JULDAYQY - Given quarter and year gets julian value. JULDAYY - Given year gets julian value. MAKEJUL - Make a Julian date from a time series Estimation and residual testing Subroutines ARMA - ARMA estimation using ML and MOM. AUTOBJ - Automatic Estimation of Box-Jenkins Model BDS - BDS Nonlinearity test. BESTREG - Best OLS REGRESSION BPFILTER - Baxter-King Filter. CANCORR - Calculate Canonical Correlations DF - Calculate Dickey-Fuller Unit Root Test. FLS - Flexable Least Squares option FLSGRAPH - Graph FLS Results FLSFRONT - Graph FLS Model Frontier GAMFIT - Generalized Additive Model Estimation GAMFORE - Forecast a GAM Model GAMPLOT - Display GAMFIT Results GARCHEST - Estimate ARCH/GARCH model. IRF - Impulse Response Functions of VAR Model KEENAN - Keenan Nonlinearity test KSWTEST - K Period Stock Watson Test KSWBOOTS - Critical values for output from KSWTEST KSWTESTM - Moving Period Stock Watson Test LAGTEST - 3-D Graph to display RSS for OLS Lags LAGTEST2 - 3-D Graph to display Rss for MARS Lags LM - Engle Lagrange Multiplier ARCH test. MINIMAX - Estimate MINIMAX with MAXF2 OLSQ - Estimate OLS, MINIMAX and L1 models. OLSPLOT - Plot of Fitted and Actual Data & Res POLYFIT - Fit an nth degree polynomial POLYVAL - Evaluate an nth degree polynomial PP - Calculate Phillips Peron Unit Root test PROBIT - Estimate Probit (0-1) Model. QUANTREG - Quantile Regression Program RESET - Calculate Ramsey(1969) regression specification test. RESET77 - Thursby - Schmidt Regression Specification Test REVERSE - Test a real*8 vector for reversibility in Freq. Domain ROTHMAN - Test a real*8 vector for reversibility in Time Domain RRPLOTS - Plots Recursive Residual Data RRPLOTS2 - Plots Recursive Residual Coef SIMULATE - Dynamically Simulate OLS Model SMOOTH - Do exponential smoothing. SWARTEST - Stock-Watson VAR Test STEPWISE - Stepwise OLS Regression TRIPLES - Calculate Triples Reversability Test TSAY - Calculate Tsay nonlinearity test. TSD - Interface to TSD Data set Estimation and residual testing and Spline Functions BOOTI - Calculate integers to be used with bootstrap. BOOTV - Bootstraps a vector with replacement. BOXCOX - Box-Cox Transformation of a series given lamda. MLSUM - Sums log of elements of a 1d object. Spline Related Commands ACEFIT - Alternating Conditional Expectation Model Estimation BSNAK - Compute Not a Knot Sequence BSOPK - Compute optimal spline knot sequence BSINT - Compute 1-D spline interpolant given knots BSINT2 - Compute 2-D spline interpolant given knots BSINT3 - Compute 3-D spline interpolant given knots BSDER - Compute 1-D spline values/derivatives given knots BSDER2 - Compute 2-D spline values/derivatives given knots BSDER3 - Compute 3-D spline values/derivatives given knots BSITG - Compute 1-D spline integral given knots BSITG2 - Compute 2-D spline integral given knots BSITG3 - Compute 3-D spline integral given knots CSPLINEFIT - Fit a 1 D Cubic Spline using alternative models CSPLINE - Calculate a cubic spline for 1 D data CSPLINEVAL - Calculate spline value given spline CSPLINEDER - Calculate spline derivative given spline value CSPLINEITG - Calculate integral of a cubic spline GAMFIT - Generalized Additive Model Estimation GAMFORE - Forecast a GAM Model GAMPLOT - Display GAMFIT Results MARS - Multivariate Autoregressive Spline Models MARSPLINE - Updated MARS Command using Hastie-Tibshirani code MARS_VAR - Joint Estination of VAR Model using MARS Approach PISPLINE - Pi Spline Nonlinear Model Building Integration Subroutines DQDAG - Integrate a function using Gauss-Kronrod rules DQDNG - Integrate a smooth function using a nonadaptive rule. DQDAGI - Integrate a function over infinite/semi-infinite interval. DQDAGP - Integrate a function with singularity points given DQDAGS - Integrate a function with end point singularities DQAND - Multiple integration of a function DTWODQ - Two Dimensional Iterated Integral IO routines Subroutines BACKSPACE - Backspace a unit CLOSE - Close a logical unit. COPYLOG - Copy file to log file. COPYOUT - Copy file to output file. COPYF - Copy a file from one unit to another. DMFGET - Load data froma DMF Save file DMFPUT - Place data in a DMF save file DMFMERGE - Merge two DMF save files ECHOOFF - Turn off listing of execution. ECHOON - Turn on listing of execution. EPPRINT - Print to log and output file. EPRINT - Print to log file. ERASE - Erase file(s). FORMS - Build Control Forms FPRINT - Formatted print facility. GET - Gets a variable from b34s. GETDMF - Gets a data from a b34s DFM file. GETKEY - Gets a key GETMATLAB - Gets data from matlab. GETRATS - Reads RATS Portable file. GETSCA - Reads SCA FSAVE and portable portable files HEADER - Turn on header ISEXTRACT - Place data in a structure. LOADDATA - Load Data from b34s into MATRIX command. MAKEDATA - Place data in a b34s data loading structure. MAKEDMF - Place Data in a b34s DMF file MAKEFAIR - Make Fair-Parke Data Loading File MAKEMAD - Makes SCA *.MAD datafile from vectors MAKEMATLAB - Place data in a file to be loaded into Matlab. MAKERATS - Make RATS portable file. MAKESCA - Make SCA FSV portable file. MENU - Put up user Menu for input MESSAGE - Put up user message and allow a decision. NOHEADER - Turn off header OPEN - Open a file and attach to a unit. PRINT - Print text and data objects. PRINTALL - Lists all variables in storage. PRINTOFF - Turn off Printing PRINTON - Turn on Printing (This is the default) READ - Read data directly into MATRIX workspace from a file. RESTORE - Load data back in MATRIX facility from external save file. REWIND - Rewind logical unit. RMATLAB - Runs Matlab SAVE - Save current workspace in portable file format. TABULATE - List vectors in a table. WRITE - Write an object to an external file. Matrix Functions DERIVATIVE - Analytic derivative of a vector. DET - Determinate of a matrix. DIAG - Place diagonal of a matrix in an array. DIAGMAT - Create diagonal matrix. EIGENVAL - Eigenvalue of matrix. Alias EIG. INV - Inverse of a real*8 or complex*16 matrix. KPROD - Kronecker Product of two matrices. PDFAC - Cholesky factorization of PD matrix. PDFACDD - Downdate Cholesky factorization. PDFACUD - Update Cholesky factorization. PDINV - Inverse of a PD matrix. PDSOLV - Solution of a PD matrix given right hand side. PERMUTE - Reorder Square Matrix PINV - Generalized Inverse. PROD - Product of elements of a vector. QR - Obtain Cholesky R via QR method using LAPACK. QRFAC - Obtain Cholesky R via QR method. QRSOLVE - Solve OLS using QR. RCOND - 1 / Condition of a Matrix. ROLLDOWN - Moves rows of a 2d object down. ROLLLEFT - Moves cols of a 2d object left. ROLLRIGHT - Moves cols of a 2d object right. ROLLUP - Moves rows of a 2d object up. SCHUR - Performs Schur decomposition SEIGENVAL - Eigenvalues of a symmetric matrix. Alias SEIG. SVD - Singular value decomposition of an object. TRACE - Trace of a matrix. TRANSPOSE - Transpose of a matrix. ZDOTC - Conjugate product of two complex*16 objects. ZDOTU - Product of two complex*16 objects. Miscellaneous Subroutines BREAK - Set User Program Break Point. COMPRESS - Compress workspace. COPY - Copy an object to another object LAPACK - Sets Key LAPACK parameters LOAD - Load a Subroutine from a library. MAKEGLOBAL - Make a variable global (seen at all levels). MAKELOCAL - Make a variable seen at only local level. MANUAL - Place MATRIX command in manual mode. NAMES - List names in storage. LRE - McCullough Log Relative Error PCOPY - Copy an object from one pointer address to another REAL16INFO - Obtain Real16 info REAL16OFF - Turn off Real16 add REAL16ON - Turn on extended accuracy REAL32OFF - Turn off Real32 add REAL16ON - Turn on Real*32 extended accuracy RUN - Terminates the matrix command being in "manual" mode. SETLEVEL - Set level. SETWINDOW - Set window to main(1), help(2) or error(3). SIGD - Set print digits. Default g16.8 STOP - Stop execution of a matrix command. TESTARG - Lists what is passed to a subroutine or function. TIMER - Gets CPU time. VOCAB - List built-in subroutine vocabulary. Miscellaneous Functions ARGUMENT - Unpack character argument at run-time FACT - Factorial LEVEL - Returns current level. POINTER - Machine address of a variable. POLYROOT - Solution of a polynomial. POLYDV - Division of polynomials. POLYMULT - Multiply two polynomials VOCAB - List built in functions. Nonlinear Estimation Capability Subroutines CMAXF1 - Constrained maximization of function using zxmwd. CMAXF2 - Constrained maximization of function using dbconf/g. CMAXF3 - Constrained maximization of function using db2pol. BGARCH Calculate function for a BGARCH model. GARCH - Calculate function for a ARCH/GARCH model. GARCHEST - Estimate ARCH/GARCH model. LPMAX - Solve Linear Programming maximization problem. LPMIN - Solve Linear Programming minimization problem. MAXF1 - Maximize a function using IMSL ZXMIN. MAXF2 - Maximize a function using IMSL DUMINF/DUMING. MAXF3 - Maximize a function using simplex method (DU2POL). NLEQ - Jointly solve a number of nonlinear equations. NLLSQ - Nonlinear Least Squares Estimation. NL2SOL - Alternative Nonlinear Least Squares Estimation. NLPMIN1 - Nonlinear Programming fin. diff. grad. DN2CONF. NLPMIN2 - Nonlinear Programming user supplied grad. DN2CONG. NLPMIN3 - Nonlinear Programming user supplied grad. DN0ONF. NLSTART - Generate starting values for NL routines. QPMIN - Quadratic Programming. SOLVEFREE - Set frequency of freeing temp variables. Random Number Generation and Testing - IMSL Names preserved I_RNGET - Gets seed used in IMSL Random Number generators. I_RNSET - Sets seed used in IMSL Random Number generators. I_RNOPG - Gets the type of generator currently in use. I_RNOPT - Selects the type of uniform (0,1) generator. I_DRNSES - Initializes the table used by shuffled generators. I_DRNGES - Get the table used in the shuffled generators. I_DRNUN - Uniform (0,1) Generator I_DRNNOR - Random Normal Distribution I_RNBIN - Random integers from binomial distribution I_DRNGDA - Random integers from discrete distribution alias approach I_DRNGDT - Random integers from discrete using table lookup I_RNGEO - Random integers from Geometric distribution I_RNHYP - Random integers from Hypergeometric distribution. I_RNNBN - Negative binomial distribution I_DRNBET - Random numbers from beta distribution I_DRNCHI - Random numbers from Chi-squared distribution I_DRNCHY - Random numbers from Cauchy distribution I_DRNEXP - Random numbers from standard exponential I_DRNEXT - Random numbers from mixture of two exponential distributions I_DRNGAM - Random numbers from standard gamma distribution I_DRNGCT - Random numbers from general continuous distribution I_DRNLNL - Random numbers from lognormal distribution I_DRNNOA - Random normal numbers using acceptance/rejection I_DRNNOR - Random normal numbers using CDF method I_DRNSTA - Random numbers from stable distribution I_DRNTRI - Random numbers from triangular distribution I_DRNVMS - Random numbers from Von Mises distribution I_DRNWIB - Random numbers from Weibull distribution I_RNMTN - Random numbers from multinomial distribution I_DRNMVN - Random numbers from multivariate normal I_DRNSPH - Random numbers on the unit circle I_RNPER - Random perturbation of integers I_RNSRI - Index of random sample without replacement Screen I/O and Plot Subroutines CLS - Clear screen. FPLOT - Plot a Function GRAPH - High Resolution graph. GRAPHP - Multi-Pass Graphics Programing Capability GRCHARSET - Set Character Set for Graphics GRREPLAY - Graph replay and reformat command. OUTDOUBLE - Display a Real*8 value at a x, y on screen. OUTINTEGER - Display an Integer*4 value at a x, y on screen. OUTSTRING - Display a string value at a x, y point on screen. PLOT - Line-Printer Graphics SCREENCLOSE - Turn off Display Manager SCREENOPEN - Turn on Display Manager SCREENOUTOFF - Turn screen output off. SCREENOUTON - Turn screen output on. Statistical Functions BETAPROB - Calculate a beta probability. BINDF - Evaluate Binomial Distribution Function BINPR - Evaluate Binomial Probability Function BJ_IDEN - ACF and PACF Listing and or Plotting BLUS - BLUS Residual Analysis CHISQPROB - Calculate chi-square probability. DERF - Error function of real*8/real*16 argument. DERFC - Inverse of error function. FPROB - Probability of F distribution. HYPDF - Evaluate Hypergeometric Distribution Function HYPPR - Evaluate Hypergeometric Probability Function HC_SIGMA - Het-Consistent estimate of Sigma INVBETA - Inverse beta distribution. INVCHISQ - Inverse Chi-square distribution. INVFDIS - Inverse F distribution. INVTDIS - Inverse t distribution. MEAN - Average of a 1d object.vector. MEDIAN - Median of a real*8 object. NCCHISQ - Non central chi-square probability. NORMDEN - Normal density. POIDF - Evaluate Poisson Distribution Function POIPR - Evaluate Poisson Probability Function PROBIT - Inverse normal distribution. PROBNORM - Probability of normal distribution. PROBNORM2 - Bivariate probability of Normal distribution. Q1 - Q1 of a real*8 object Q3 - Q3 of a real*8 object SC_TEST - Serial Correlation Tests TDEN - t distribution density. TPROB - t distribution probability. VARIANCE - Variance of an object. System Subroutines DODOS - Execute a command string if under dos/windows. DOUNIX - Execute a command string if under unix. SYSTEM - Issue a system command. Time Series Functions ACF - Calculate autocorrelation function of a 1d object. CCF - Calculate the cross correlation function on two objects. DIF - Difference a series. FRACDIF - Fractional Differencing. GENARMA - Generate an ARMA series given parameters. MAKEJUL - Make a Julian date from a time series Variable Precision Math Subroutines and Functions Subroutines VPASET - Set Variable Precision Math Options Functions VPA - Variable Precision Math calculation Listing of Alias function names Name Replaced by LOG DLOG LN DLOG LOG10 DLOG10 EXP DEXP MOD DMOD MAX DMAX MAX1 DMAX1 MIN DMIN MIN1 DMIN1 INVERSE INV SIN DSIN COS DCOS R4TOR8 DFLOAT R8TOR4 FLOAT SQRT DSQRT SINH DSINH GAMMA DGAMMA COSH DCOSH CONJ DCONJ ATAN DATAN ATAN2 DATAN2 ARSIN DARSIN ARCOS DARCOS ABS DABS R8TOR4 SNGL R4TOR8 DBLE Index of Toolkit Routines contained in matrix2.mac Index of programs, subroutines and Functions ACF_PLOT - Simple ACF Plot AUTOCOV - Autocovariance BPF - Baxter - King MA Filter BPFM - Baxter - King MA Filter with missing data CFREQ - Determine Cumulative Frequency Distribution CATS - Cointegration Test using Johansen Method COINT2 - Cointegration Tests of Two Series COINT2LM - Cointegration Tests of Two Series, OLS, L1, MM COINT2M - Moving Cointegration of Two Series COINT2ME - Moving Cointegration of Two Series - Extended Args. COINT2M2 - Moving Cointegration of Two Series OLS, L1, MM COINT3 - Cointegration Tests of Three Series COINT3ME - Moving Cointegration of Three Series COINT_SW - Stock-Watson DOLS Analysis DATA_ACF - Calculate ACF and PACF Plots DATA2ACF - Calculate ACF and PACF Plots added argument DATAVIEW - View a Series Under Menu Control DO_SPEC - Display Periodogram and Spectrum DO2SPEC - Display Periodogram and Spectrum added argument DUD - Derivative Free Nonlinear Estimation FDIFINFO - Fractional Differencing Information FILTER - High Pass / Low Pass Filter using Real FFT FILTERC - High Pass / Low Pass Filter using Complex FFT FORPLOT - Forecast Plot using GRAPHP GARCH2P - Two Pass GARCH Using ARMA Command GARCH2PF - Two pass GARCH using ARMA Command with forecasting GTEST - Tests output of a ARCH/GARCH Model GWRITE - Save Objects in GAUSS Format using one file GWRITE2 - Save Objects in GAUSS Format using two files HP_BP_1 - Baxter-King & Hodrick-Prescott Filtering HP_BP_2 - Baxter-King & Hodrick-Prescott Filtering Moving Window HP_2 - Hodrick & Prescott Filtering of a Moving Window IRF - Impulse Response Functions of VAR Model LMTEST - Engle (1982) test for ARCH for a range of lags MARSPLOT - Automatically plot MARS Curves and Surface Plots MCLEODLI - McLeod-Li (1983) Linearity test MARQ - Estimation of a Nonlinear Model using Derivatives MINIMAX - Estimate MINIMAX with MAXF2 MISSPLOT - Plot of a series with Missing Data MQSTAT - Multivariate Q Statistic MOVEAVE - Moving average of a vector MOVEBJ - Moving Arima Forecast using AUTOBJ MOVEH82 - Moving Hinich 82 test MOVEH96 - Moving Hinich 96 test MOVEOLS - Moving OLS Calculation MOVEVAR - Moving Variance NLVARCOV - Calculates NLLSQ Variance Covariance OLSPLOT - Plot of Fitted and Actual Data & Res PAD - Pad a 1D Real*8 Series on both ends PVALUE_1 - Present value of $1 recieved at end of n years PVALUE_2 - Present Value of an Annuity of $1 PVALUE_3 - Present value of $1 recieved throughout year RTEST - Test Residuals of Model RTEST2 - Test Residuals of Model - No RES and Y Plots QUANTREG - Quantile Regression Program RESET77 - Thursby - Schmidt Regression Specification Test SUBSET - Subset 1d, 2d array, vector or matrix under a mask. Running the Matrix Command: The MATRIX command can be run in "batch mode" or interactively. When run interactively, statements can be reloaded and scripts can be edited and submitted. The file _imatrix.mac is the default name for the script files. The first member _matrix is automatically run if the script is submitted. The B34S save file format is the same as used by Speakeasy and the Speakeasy importall statement can be used to further process this file. The MATRIX command can also read and write SCA and RATS portable files and provides a bridge between these systems. The Display Manager provides a number of options on how to use the system. 1. The MATRIX button on the Display Manager window gets B34S into interactive MATRIX command mode. If the user wants to customize how the matrix command will come up, the IMATRIX macro in the MATRIX.MAC file can be changed. The user must not make an error in this file since the whole interative matrix command will be unable to execute if there is an error.. In MANUAL mode one command at a time can be entered or scripts can be edited and run. Interative use is intended for quick commands. If the IMATRIX member of the MATRIX.MAC file is modified, the MATRIX button could be made to execute a matrix program, or another B34S command BEFORE going into MANUAL mode in the matrix command. The default IMATRIX member of matrix.mac is: /$ This shell can be modified to load data set if desired b34sexec matrix; call manual; b34srun; In the user directory it is possible is the _imatrix.mac file that has a required member _matrix which can be edited and submitted while under in the interactive matrix command mode. A sample file is: ==_matrix program _matrix; /$ Lines above this are required x=array(100:); y=rn(x); z=rn(y); x=rn(x); call graph(y,z); call names(all); call tabulate(x,y,x); call olsq(y x:print); call load(user '_imatrix.mac'); call user; /$ Lines next are required call manual; return; end; == ==user program user; /$ this is a user program; call print('I am in the user program user!!':); return; end; == A simple example is: /; /; Sample user _imatrix.mac file /; ==_MATRIX program _matrix; /; Commands after here x=rn(matrix(5,5:)); ix=inv(x); call print(x,ix,x*ix); call print('How are things going'); return; end; == Remember the _imatrix.mac file is in the user directory. It submits matrix program while inside the matrix command. The imatrix macro in matrix.mac allows data to be loaded etc BEFORE calling the matrix command!! 2. The MATRIX command has been designed to run in "batch" mode from a command file. This file is usually submitted from a FILE or from the TASKS command of the Display Manager. 3. The LINEEDIT option under the MENU button allows multiple lines to be entered. This way quick MATRIX jobs can be submitted. By use of the MENU "RELOAD" facility, the commands just submitted can be recalled and changed but are not saved in memory. 4. The MENU command provides two options. The MATRIX command gives the user a screen where any MATRIX commands can be entered. In this mode a "batch file" is interactively submitted. This mode is designed for quick jobs that call user PROGRAMS, SUBROUTINES and FUNCTIONS. By use of the RELOAD option, these commands can be modified and resubmitted. The MENU command also allows access to the MATRIX command which is the same as what is available with the IMATRIX Display Manager command (see # 1 above). In the middle of a MATRIX command section the command: call save; will save the workspace. Later in the session OR at a later session, the command: call restore; will bring back what has been saved. The default name is matrix.psv A specific file can be given using the forms call save(:file 'mystuff.psv'); which can be brought back with call restore(:file 'mystuff.psv'); These save files can be read with Speakeasy(r) using the Speakeasy IMPORTALL command. If b34s matrix command PROGRAMS, SUBROUTINES or FUNCTIONS are present in the B34S MATRIX command workspace, the command call save(:speakeasy); will save only data objects. Because Speakeasy will not support variables of the form %name, such variables are name _name. On a restore these are converted back. The keyword checkpoint can be used in the place of save to be compatible with Speakeasy. If checkpoint is used, subroutines, functions and programs are automatically not saved. The MATRIX command interactive mode can be terminated by call run; and started later in the same job with call manual; This way a complex program can be debuged by looking an intermediate values. Since all output is written to the b34s output file in the usual manner, View log and View output are always available to inspect results. Help The usual b34s help facility provides a discussion of all B34S MATRIX commands. In addition the file matrix.mac provides an example of virtually all MATRIX commands. These examples can be viewed with the help facility, executed or saved in the B34S buffer where they can be either run or further modified. The file matrix2.mac contains programs, subroutines or fuctions that can be run provided that they are loaded with the command call load(somename); There are a number of ways to load data in the matrix command. These are briefly mentioned here. 1. If data is already loaded in B34S then the loaddata and get commands can be used. b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call echooff; call loaddata; call load(autocov); call autocov(gasout,aa,norows(gasout)/2); aatest=acf(gasout,norows(gasout)/2); call tabulate(aa,aatest); call graph(aatest :heading 'ACF'); call graph(aa :heading 'Autocovariance'); b34srun; A subset can be loaded as b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call get(gasout,gasin); call names; call graph(gasout); b34srun; 2. If data is in a csv file many options are available including missing data set, character data set call csv(:readfile 'mycsv.csv' ); call csv(:readfile 'mycsv.csv' ); call csv(:readfile 'mycsv.csv' :var x); call csv(:readfile 'mycsv.csv' :var x); call csv(:writefile 'mycsv.csv' :var x1 x2 x3); call csv(:writefile 'mycsv.csv' :var x1 x2 x3 :nonames :nodatestamp); The csv command can also load a blank delimited real*8 data file where the names are in line 1, and there are no missing data. Such a file is: Date rtn vwretd ewretd sprtrn 19700102 0.000686 0.012137 0.033450 0.010211 19700105 0.009596 0.006375 0.018947 0.004946 19700106 0.000679 -0.007233 -0.005776 -0.006848 19700107 0.000678 -0.001272 0.003559 -0.002047 19700108 0.002034 0.000564 0.002890 0.000540 19700109 -0.001353 -0.002797 -0.002923 -0.003021 19700112 -0.003388 -0.007766 -0.009646 -0.007576 19700113 0.017335 0.000737 -0.001090 0.000981 19700114 -0.001002 -0.002005 -0.005280 -0.001525 19700115 0.020736 0.000586 -0.002182 0.000327 19700116 -0.030799 -0.007462 -0.006902 -0.008290 19700119 -0.053414 -0.013079 -0.009019 -0.013968 19700120 0.018571 0.001938 0.002439 0.002008 19700121 -0.004208 0.001017 0.002280 0.000891 19700122 0.002817 0.001430 0.001346 0.000445 19700123 0.000000 -0.010282 -0.011356 -0.009783 Such a file can be read by R as da=read.table("test.txt",header=T) rtn =da[,2] vwretd=da[,3] ewretd=da[,4] sprtrn=da[,5] In B34S the commands are: b34sexec matrix; call csv(:readfile 'test.txt'); mm=mean(rtn); Note that the csv command does no require the user knows the number of series or the number of observations. When these values are known, the dataread command is available. 3. Subset of the capability of the dataread command: b34sexec matrix; datacards; 3 4 x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 b34sreturn; call dataread(:unit 4); call names; call tabulate(x1 x2 x3 x4); b34srun; /; Here :noob and :nvar are set, first card not needed b34sexec matrix; datacards; x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 b34sreturn; call dataread(:unit 4 :precision real8 :noob 3 :nvar 4); call names; call tabulate(x1 x2 x3 x4); b34srun; /; Reading from a file b34sexec matrix; call open(77,'test2.txt'); c=c1array(4,12: ' x1 x2 x3 ' ' 11. 22. 33.' ' 44. 55. 66.' ' 77. 88. 99.'); /; build test file do i=1,4; call write(c(i,),77); enddo; call close(77); /; Now read test file call dataread(:file 'test2.txt' :noob 3 :nvar 3); call tabulate(x1,x2,x3); b34srun; Example # 2 More Complex Examples of dataread /; b34sexec options debugsubs(readob_vpa, /; b34smatdataread); b34srun; /; Shows three reads b34sexec matrix; datacards; 3 4 x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 3 4 y1 y2 y3 y4 11 22 33 joe 55 66 77 sam 99 10 11 tom b34sreturn; call dataread(:unit 4 :precision real8); call names; call tabulate(x1 x2 x3 x4); /; Testing means call rewind(4); call dataread(:unit 4 :precision real8 :means); call print(%means,%vnames); /; Read next 4 obs call dataread(:unit 4 :character namelist(y4)); call tabulate(y1 y2 y3 y4); call rewind(4); call names(all); /; now reading first 4 in real*16 oldx1=x1; oldx2=x2; oldx3=x3; oldx4=x4; call rewind(4); call print('Reading data into real*16':); call print('_________________________':); call dataread(:unit 4 :precision real16 :means); call tabulate(x1 x2 x3 x4); call print(%means,%vnames); call names(all); call rewind(4); call names(all); /; Reading in vpa oldx1=x1; oldx2=x2; oldx3=x3; oldx4=x4; call print('Reading data into VPA':); call print('_____________________':); call dataread(:unit 4 :precision vpa :means ); call tabulate(x1 x2 x3 x4); call print(%means); call names(all); b34srun; /; Using Option # 2 b34sexec matrix; datacards; 11 22 33 44 55 66 77 88 99 10 11 12 11 22 33 joe 55 66 77 sam -9 -8 -7 tom b34sreturn; /; /; doing two reads; /; call dataread(:unit 4 :names namelist(x1 x2 x3 x4) :precision real8 :noob 3); call tabulate(x1 x2 x3 x4); call dataread(:unit 4 :character namelist(y4) :names namelist(y1 y2 y3 y4) :noob 3); call tabulate(y1 y2 y3 y4); b34srun; /; Create a file and read it back b34sexec matrix; call echooff; noob=100; nvar=3; x=rn(array(noob,nvar:)); call print(x); call open(77,'test.txt'); do i=1,noob; call write(x(i,),77); enddo; call close(77); call dataread(:file 'test.txt' :noob noob :names namelist(x1 x2 x3) :precision real16); call tabulate(x2,x2,x3); call print(mean(x(,1)), mean(x(,2)), mean(x(,3))); call print(mean(x1 ), mean(x2 ), mean(x3 )); call open(77,'test2.txt'); c=c1array(3,12: ' 11. 22. 33.' ' 44. 55. 66.' ' 77. 88. 99.'); do i=1,3; call write(c(i,),77); enddo; call close(77); call dataread(:file 'test2.txt' :noob 3 :format '(3f4.0)' :names namelist(x1 x2 x3) :precision real8); call tabulate(x1,x2,x3); b34srun; Example # 3 using dataread to make a mac file. The only limit is the B34S data step limit of 98. This example gets around 80 col limit on input. In matrix, there is effectively no variable limit. Note: call catname( ) needed if in the namelist there is a 8 character name. In this application a blank is added. b34sexec matrix; call load(catname :staging); call dataread(:file 'murder.raw' :noob 153 :nvar 13 :means :nocols 700 :names namelist(id state year mrdrte exec unem d90 d93 cmrdrte cexec cunem cexec_1 cunem_1) :character namelist( stateabb state) ); call setlabel(id ,'state identifier '); call setlabel(state ,'postal code '); call setlabel(year ,'87, 90, or 93 '); call setlabel(mrdrte ,'murder rate '); call setlabel(exec ,'total executions, past 3 years'); call setlabel(unem ,'annual unem. rate '); call setlabel(d90 ,'=1 if year == 90 '); call setlabel(d93 ,'=1 if year == 93 '); call setlabel(cmrdrte,'mrdrte - mrdrte[_n-1] '); call setlabel(cexec ,'exec - exec[_n-1] '); call setlabel(cunem ,'unem - unem[_n-1] '); call setlabel(cexec_1,'cexec[_n-1] '); call setlabel(cunem_1,'cunem[_n-1] '); call print(%means); call catname(%vnames,' ',%vnames,0); call makedata(argument(%vnames) :file 'wood.mac' :heading 'murder data' :member murder :add); b34srun; 4. For more specialized readwrite capability see: DMFGET - Load data froma DMF Save file DMFPUT - Place data in a DMF save file DMFMERGE - Merge two DMF save files GETDMF - Gets a data from a b34s DFM file. GETMATLAB - Gets data from matlab. GETRATS - Reads RATS Portable file. GETSCA - Reads SCA FSAVE and MAD portable files MAKEDATA - Place data in a b34s data loading structure. MAKEDMF - Place Data in a b34s DMF file MAKEFAIR - Make Fair-Parke Data Loading File MAKEMAD - Makes SCA *.MAD datafile from vectors MAKEMATLAB - Place data in a file to be read by Matlab. MAKERATS - Make RATS portable file. MAKESCA - Make SCA FSV portable file. Form of MATRIX command. b34sexec matrix options parameters; b34seend$ MATRIX Command Options SAVEASVECTOR - Saves B34S variables as vectors. Default is to save as an array. For the differences between vector and array math, see below. For a minor example, if the b34s variable GASOUT is saved as an array. The statement test=gasout**2.; will square each element, If GASOUT is saved as a vector, the command test=gasout**2.; will produce one observation which is the sum of squared values of GASOUT. SHOWUSE - Shows use of arrays. Indicates current limits of the program. HEADER - Turns on Header. Default is NOHEADER. The commands call noheader; call header; can turn the header off and on inside the matrix step. CBUFFER=n1 - Sets size of Command buffer. If a bigger size is needed, a message will be given. For further info see below. SIGD=n2 - Sets number of significant digits in variable printing. Default g16.8 for real*8. This can also be set by call sigd(i); in open matrix language code. If i le 8 g16.i is used. If i > 8 then g16+(i-8).i format is used. For example i = 10 uses g18.10. SOLVEFREE=n3 - Sets frequency of cleaning temp variables with SOLVE command. Default = 50. If error message on exceeding temp variables is seen, set number smaller. A larger n2 value can reduce CPU time. The subroutine SOLVEFREE can be used to test your code. DSEED=r - Sets the seed for GGNML and GGUBS and other routines. This seed works the same but can be set to a different value than the seed set under the OPTIONS command. The exact random number routine used is set on the OPTIONS command with the command RECVER and RNVER. Consult help files for this command for further detail. Default = 123457.0d+00 or the seed set with SETSEED on the OPTIONS command. The possibly of a different seed for the MATRIX command isolates how this command runs in relation to a default seed set with SETSEED in a user AUTOEXEC.B34 file. Since not all random number generators are equal, users should use caution. Empirically it was found that if 3000 random normal deviates are generated using the default (IMSL) seed, the BDS statistic flags the series as "nonlinear." This finding is strange to say the least. Using I_RNOPT command the exact uniform generator for IMSL random number routines can be selected. The MATRIX command RN( ) and REC( ) have optional switches that allow the default random number routines to be replaced by IMSL version 10 routines. DISPLAY=key - Sets accuracy and number of Cols to print matrix output. Key can be set: COL80FIXED f18.5 COL80MEDIUM g15.6 COL80HIGH g25.16 COL129FIXED f18.5 COL129MEDIUM g15.6 COL129HIGH g25.16 The default is COL129MEDIUM. This can be set globally with the options commands linesize and sigd LINESIZE SIGD DISPLAY 80 < 8 COL80FIXED 80 8 COL80MEDIUM 80 > 8 COL80HIGH 132 < 8 COL129FIXED 132 8 COL129MEDIUM 132 > 8 COL129HIGH ADJUST_PARSE - Can be used if functions are being called outside subroutines only. For more detail see the FUNCTION help file. This option is not the the general user. MAXVAR=i4 - Sets maximum number of variables in OLSQ, MARSPLINE etc. Default = 400. Upper limit = 999. Lower limit =10. Size of problems: The maximum size of any problem is limited by the maximum number of sentences in the parse table (MAXNSENT) and the maximum number of tokens (MAXNTOKEN). These can be increased in the autoexec.b34 file on the PC, or in an OPTIONS command such as b34sexec options maxnsent(600) maxntoken(5000); b34srun; Other current constraints are: Max number of objects in memory 10000. Max number of arguments to a function or subroutine 2500. These constraints may change in the future. For other limitations see the SHOWUSE command of the MATRIX sentence. Overview of the Matrix Language Cont. The goal of the MATRIX command is to provide a high-level programing language that will allow the user to easily customize a B34S application. The MATRIX command consists of analytic statements that that can be reduced to assingment statements and keywords. In contrast to address oriented programing languages such as FORTRAN, the B34S matrix language is object oriented. Assuming X was a vector of 20 numbers, the command ameanx=mean(x); places the mean of the x vector in the variable ameanx. Keywords such as PRINT are used in the form call print('This will print x',x); which will print the statement and the object x. The PRINT command will allow up to 400 arguments or objects to be passed in one call. Keywords are built into the language or can be user written SUBROUTINES, FUNCTIONS or PROGRAMS. User SUBROUTINES and FUNCTIONS pass objects as arguments and calculate in a protected workspace. This means that a variable X can be defined at both the base level and in the SUBROUTINE to mean different things. The difference between PROGRAMS and SUBROUTINES is that SUBROUTINES pass arguments and work in a protected space while PROGRAMS do not pass arguments. PROGRAMS use the current level as their name domain. Hence if a PROGRAM is called from the base level, it can access objects known at that level. If the same programn is called from a SUBROUTINE, it will access data known at that level only. User SUBROUTINES and FUNCTIONS also have access to GLOBAL variables. Recursive Notes: User SUBROUTINES and FUNCTIONS can be called recursively up to the MAXSTAK limit currently 5000, although CBUFFER will have to be increased to handle saving the open arguments. Recursive calls are very very slow but are useful in some situations. Job RECURSIVE in matrix.mac shows that for the same task, a DO loop is 100 times faster than a recursive function or subroutine call. A DO loop itself is slow relative to object calculations. FORMULA & SOLVE statements are from 4 - 10 times faster than DO loops. The speed gain depends on: 1. the length of the vectors, 2: the complexity of the problem and 3. the SOLVEFREE setting which sets how frequently temp variables are freed in the workspace. If recursive calls are a major problem, a brach to an externallu compiled Fortran program might be a good solution. For further detail, see section 14 of this help file. User SUBROUTINES, PROGRAMS and FUNCTIONS must have names of 8 or less characters. A simple example of a user subroutine is DESC subroutine desc(name,mean1,var); mean1=mean(name); var=variance(name); return; end; The following code fragment uses DESC x=array(5:1 2 3 4 5); call desc(x,m,v); call print(Variable mean var',x,m,v); A PROGRAM for the same result would be program desc2; m=mean(x); v=variance(x); return; end; The following code fragment uses DESC2. x=array(5:1 2 3 4 5); call desc2; call print(Variable mean var',x,m,v); Note that in this case the actual names are needed to be used inside DESC2 since the name domain is at the global level. A simple user function DESF will return the mean although in this case it makes little sense since we have the built-in function mean. function desf(x); m=mean(x); return(m); end; The following code fragment uses desf x=array(5:1 2 3 4 5); amean=desf(x); MATRIX command built-in FUNCTIONS and SUBROUTINES in addition to allowing name arguments, also allow passing character strings subject to a limitation shown below. Given subroutine printit(c); call print(c); return; end; If the string is LE 8 characters the form call printit('less8'); can be used. If the string is greater that 8, then the correct form is call character(c,'This is greater than 8'); call printit(c); must be used. This is illustrated in the job char_4 in the matrix.mac file. What is happening is that the form call printit('aa'); places the string 'aa' in a character*8 variable. The form call character(cc,'aa'); call printit(cc); places the string 'aa' in a two element character*1 array. Thus the statement name='J'; creates a character*8 variable. If this is used in an assingment statement to a character*1 variable, the character*1 variable will be redefined which may not be what is wanted. This is illustrated in the job char_5 which is listed next: b34sexec matrix; call echooff; x=rn(matrix(30,30:)); call character(ii,'Element (1, '); call character(ii2,' '); jj=integers(12,17); do i=1,30; call inttostr(i,ii2,'(i6)'); ii(jj)=ii2(jj-11); call character(rp,')'); /$ ********************************************************** /$ Warning the statement /$ ii(18)=')'; /$ does not work since it will be redefined to be character*8 /$ and will be outside the 132 range and not printed /$ ********************************************************** ii(18)=rp; call print(ii,x(1,i) :line); next i; call names(all); b34srun; Summary. Since the MATRIX command supports character*8 and character*1 data added care must be used to pass just what is desired to a user routine. Built-in subroutines and functions can detect just what has been passed and take the appropriate path. The command character or char1 can be used to build a multi-line 2D character*1 array. call character(text,'This is line one' 'This is line two which is longer' 'This is line three' 'This is 4'); produces a 2d character*1 array of size 4,132 where 132 is the max length of a line. Strings to the character command can be up to 132 in length. Advanced concepts. Note: If a MATRIX command variable name is passed to a user SUBROUTINE or FUNCTION, it is possible to return a changed value. If a structured object is passed, it is NOT possible to return a value. This follows the Speakeasy convention but is not the usual case in Fortran. The reason for this is that the structured object may repackage the data, which is not the case in Fortran. For example assume matrix A defined as n=10; a=rn(matrix(n,n:)); if we call desc (see routine listed above) as call desc(a(1,),m,v); mm(1)=m; vv(1)=v; call desc(a(2,),m,v); mm(2)=m; vv(2)=v; we have the desired result since DESC does not change the first argument. However the code call desc(a(1,),m(1),v(1)); call desc(a(2,),m(2),v(2)); will not work as intended because in the parsing process, m(1), m(2), v(1), and v(2) are replaced by temporary variables. After returning the temp values not the real values are saved. One would think that this could be taken care of by the compiler BUT this is not possible. The reason relates to how stuctured objects are handled. Consider the following code. x=rn(matrix(10,10:)); row2=x(2,); In the matrix the elements of row 2 differ by 10 memory positions since Fortran saves by column. However the command row2=x(2,); "repackages" these elements so that they are next to each other. Hence putting them back into the matrix X is not just a copy operation. For this reason users have to be careful when passing structured objects. B34S MATRIX USER SUBROUTINES and FUNCTIONS pass structured objects by value NOT by address! More detail on this is provided below. In other languages such as MATLAB arguments cannot be changed at all. In MATLAB if the user has a function myfun that takes three arguments and returns two, the MATLAB calling form is [back1,back2]=myfun(in1,in2,in3); where in matlab the function is defined by function[bb,cc]=myfun(aa1,aa2,aa3) The B34S MATRIX command follows the Speakeasy conventions and allows a bit more flexibility. However users have to be careful. The MATRIX language recognizes the following types of objects which are given kind and klass numbers. kind character*1 -1 real*4 4 real*8 8 real*16 -16 complex*16 16 complex*32 32 program 1 subroutine 2 function 3 formula 33 character*8 -8 integer*4 -4 integer*8 -88 vpa real 88 vpa real packed 888 vpa integer -44 vpa integer packed -444 vpa complex 160 vpa complex packed 1600 not defined -99 Klass scalar 0 vector 1 matrix 2 1 dim array 5 2 dim array 6 The matrix language recognizes structured objects. Define a as matrix 1. 2. 3. 4. 5. 6. 7. 8. 9. with the command a=matrix(3,3:1 2 3 4 5 6 7 8 9); The matrix, array and vector commands are the only ones that automatically convert integer input to real*8. If an integer*4 matrix is desired use a=idint(matrix(3,3:1 2 3 4 5 6 7 8 9)); The command call print(a(2,)); prints row 2 while call print(a(,3)) prints column 3. After defining an integer vector i of two elements 1 and 3 i=integers(1,3,2); the commands call print('This is row 1 and 3',a(i,)); call print('This is col 1 and 3',a(,i)); will pull off rows 1 & 3 and columns 1 & 3 respectively. Remember structured objects pass as values NOT addresses. Structured objects can be used as arguments but cannot be used for output. On the left of an = sign assingments are possible. This is contrary to Fortran which passes by address. B34S "repackages" the structured object data and passes the values leaving the original array intact. Internally the matrix is saved by columns following Fortran. Hence both x(2,) and x(,2) pass contiguous values although the underlying storage is NOT contiguous for x(2,). What is happening is that the B34S parser "repackages" the second row, which was not saved as contiguous values, into a vector. For this reason it is not possible to return back values from a structured variable address such as x(,i), x(i) or x(3,) etc. In a sense this "limitation" of returning values in a structured object frees the user from having to remember the underlying storage of the matrix object. Here the vector is an object in its own right. That is how we think of it in mathmatics. The goal of the MATRIX design is to think in mathematics, not in computer storage conventions. Speed design considerations: Code such as i=integers(2,20); j=i-1; a(j) = b(i); involves repetitive expansion of A which has not been defined to the correct size and therefore has to be expanded on the fly. While such code will work, a better approach is b34sexec matrix; b=rn(array(30:)); i=integers(2,20); j=i-1; a=array(dmax(j):); a(j) = b(i); call tabulate(a,b); b34seend$ which will run substantially faster. The following code illustrates some of these concepts. x=rn(matrix(3,3:)); a(1,1)= mean(x); v=vector(3:1 2 3); a(2,)=v; i=idint(array(2:1,3)); * place terms 1 3 of v in newv; newv=v(i); x=matrix(3,3:integers(1,9)); * places rows 1 and 3 of x or 1 2 3 7 8 9 in newx; newx=x(i,); The mean of x is now in a(1,1), 1 2 3 is in row 1 of newx and 7 8 9 are in row 2. 1.0 and 3.0 are in variable newv. Data can be placed in structured objects a number of ways. To put 6.0 in col 3 of x at all locations: call setcol(x,3,6.0); To put 7.0 in row 6 of x at all locations: call setrow(x,6,7.0); another way would be x(6,)=(array(nocols(x):)+7.0); assuming x is a 2d array. Or x(6,)=7.; Object expansion is possible. The command a(6,)=v; would place v in a new row 6 of x. Due to possible ambiguity, if v is a vector and i and j are vectors, the statement x(i,j)=v; is not allowed. The command newx=matrix(n,n:)+8.0; will build a n by n matrix with 8.0 on the diagonal while newx=array(n,n:)+8.0; will put 8.0 in all elements. These statements illustrate the difference between "matrix math" and "array math." The operators .EQ. .LT. .GT. .NE. .GE. .LE. .OR. .AND. can be used in expressions as well as IF and WHILE statements. Given x=2.0; y=3.0; test1=x.eq.y; test2=x.ne.y; returns test1=0.0 and test2=1.0 If x=array(:1 2 3); y=array(:1 2 4); the commands test1=x.eq.y; test2=x.ne.y; imply test1=array(:1.0 1.0 0.0) since x(3) ne y(3) test2=array(:0.0 0.0 1.0) since x(1)=y(1) and x(2)=y(2) Note: That in an IF statement the arguments must be scalars while analytic statements of the form x= will create a scalar or vector depending on the inputs. If one input is a vector, the other input must be a scalar or a vector of the same size. Warning: The logical operators .and. and .or. can be used to compare 0.0 and 1.0 values only. Names of Objects and commands. B34S MATRIX commands that do not return an argument across an equals are executed by the CALL sentence. The CALL sentence first looks in named storage for a routine with this name. If this is not found, then the built in routines are used. While it is possible to have a user routine with the same name as a built in routine, this is not a good idea. For example assume you have loaded the series GASOUT. The command acf=acf(gasout,24); will place 24 autocorrelations of the GASOUT series in the structured variable ACF. The result is that the ACF command is no longer available. If the command acf2=acf(gasin,24); is given later in the job, B34S will object that ACF is not a 2D object. The solution is to use FREE to "turn on" the built-in command ACF. call free(acf); acf2=acf(gasin,24); or better still resist using an internal command name as a variable name. Math using the MATRIX Command. The MATRIX command allows math between 1 and 2 dimensional arrays and between vectors (1 dimensional) and matrices (2 dimensional) for real*8 and complex*16 objects. Array math can be done between real*8, real*4, real*16, integer*4 and complex*16 objects. Objects of different types cannot be mixed. If this is attempted, B34S will give a "mixed mode" error message. The reason for this is given below. Warning. Only in the most extreme cases should variables be saved as real*4. The real*4 data type is for some graphic applications and for variable storage in cases where memory is low. While array and matrix math is possible for real*4, other functions such as sin will not work to prevent users from making calcualtios that have accyracy loss. Matrix and vector math can be done only with real*4 real*8, real*16, complex*16, complex*32 variables and vpa real and complex objects. Assume r8 is real*8. newr=r8*2.0; is allowed but newr=r8*2; is not allowed since 2 is an integer. The error message will refer to "mixed mode" operations. The logic behind not allowing r8*2 is that it is not clear whether an integer*4 or real*8 result is desired. This convention is not followed in Speakeasy which tries to make everything real*8. The reason for this "restriction" is illustrated by the following example. Assume the commands x=10.; y=x*2; where given. The parser would not know if Y should be integer*4 20 or real*8 20. Note: The functions DBLE, SNGL, DINT, IDINT, INT, FLOAT, DFLOAT, REAL, IMAG, COMPLEX, r8tor16, r16tor8, c16toc32, c32toc16 i4toi8, i8toi4 and vpa can be used to convert the storage of a variable. Speed considerations: To increase run-time speed the CALL command is used in place of just naming the SUBROUTINE as is done with some other programming systems such as Speakeasy and MATLAB. B34S follows the Fortran convention. Arguments passed to subroutines and functions must be inside ( ). Built in Matrix Commands called with CALL must not be mixed. For example call names; is allowed. but call names names(all); is not allowed since the command NAMES(ALL) is not the first key word in the call sentence. Matrix Command Files: The file c:\b34slm\matrix.mac contains a number of complete matrix command examples. These can be executed, loaded into the program buffer under TASKS, or viewed using the help facility. There are working examples for virtually every matrix command. Users should customize these examples. The file c:\b34slm\matrix2.mac contains matrix command SUBROUTINES, PROGRAMS and FUNCTIONS. These can be loaded as run time with the command call load(somename); or loaded into the program buffer, or viewed with the help facility. Each file cannot contain anything except, SUBROUTINES, PROGRAMS or FUNCTIONS. The advantage of loading a routine with a CALL LOAD statement, rather than having the routine in the command file, is that parse space is saved and execution speed is also increased. If the user adds routines to the library, it is important that the code be "clean" since the parser will not check the code in the same manner as would be the case if the routine were loaded. Unless the routine produces output, most routines are called inside CALL ECHOOFF; CALL ECHOON; statements so that the statements executed will not echo to the output file. The files staging.mac and staging2.mac are like matrix.mac and matrix2 except that they are for prospective commands. These commands are documented inside the routine and not in the b34shelp.dat file. Basic rules of the B34S Matrix Command: 1. All matrix statements MUST end in $ or ;. x=dsin(q); 2. Mixed mode math is not allowed. For example assuming x is real*8 x=x*2; is not allowed because x is real*8 and 2 is an integer. The reason mixed mode is not allowed is that the processor would not know what to do with the result. The correct form is x=x*2.; or x=idint(x)*2; if you want an integer result and x was real*8 before the command. 3. Calculated Structured objects can only be used on the right of an expression or in a subroutine call as input. For example mm=mean(x(,3)); calculates the mean of col 3 while nn=mean(x(3,)); calculates the mean of row 3. i=integers(2,30); y(i)=x(i-1); copies x(1),...,x(29) to y(2),...,y(30). The code below is not allowed since it involves a calculated left hand side i=integers(1,29); y(i-1)=x(i); 4. Structured objects can be used on the left of an assignment statement to load data. The commands x=3.0; x(2)=4.0; add another element to x. x=rn(matrix(4,4:)); x(,2)=0.; places 0.0 in col 2 while x(3,)=99.; places 99.0 in row 3. The following code shows advanced structured index processing. This code is available in matrix.mac as overview_2 /$ Illustrates Structural Index Processing b34sexec matrix; x =rn(matrix(6,6:)); y =matrix(6,6:); yy =matrix(6,6:); z =matrix(6,6:); zz =matrix(6,6:); i=integers(4,6); j=integers(1,3); xhold=x; hold=x(,i); call print('cols 4-6 x go to hold',x,hold); y(i, )=xhold(j,); call print('Rows 1-3 xhold in rows 4-6 y ',xhold,y); y=y*0.0; j2 =xhold(j,); y(i, )=j2 ; call print('Rows 1-3 xhold in rows 4-6 y ',xhold,y); z(,i)=xhold(,j); call print('cols 1-3 xhold in cols 4-6 z ',xhold,z); j55 =xhold(,j); z=z*0.0; z(,i)=j55; call print('cols 1-3 xhold in cols 4-6 z ',xhold,z); yy=yy*0.0; yy(i,)=xhold; call print('rows 1-3 xhold in rows 4-6 yy',xhold,yy); zz=zz*0.0; do ii=1,3; jj=ii+3; zz(,jj)=xhold(ii,); enddo; /; i=integers(4,6); /; j=integers(1,3); call print('Note that zz(,i) = xhold(j,) will not work':); call print('Testing zzalt(,i)= transpose(xhold(j,))':); /; Use of Transpose speeds things up over do loop zzalt=zz*0.0; zzalt(,i)= transpose(xhold(j,)) ; call print('rows 1-3 xhold in cols 4-6 zz',xhold,zz,zzalt); zz=zz*0.0; zzalt=zz; do ii=1,3; jj=ii+3; zz(jj,)=xhold(,ii); enddo; call print('Note that zz(i,) =xhold(,j) will not work':); call print('Testing zzalt(i,)= transpose(xhold(,j))':); zzalt(i,)=transpose(xhold(,j)); call print('cols 1-3 xhold in rows 4-6 zz',xhold,zz,zzalt); oldx=rn(matrix(20,6:)); newx= matrix(20,5:); i=integers(4); newx(,i)=oldx(,i); call print('Col 1-4 in oldx goes to newx',oldx,newx); oldx=rn(matrix(20,6:)); newx= matrix(20,5:); i=integers(4); newx(1,i)=oldx(1,i); call print('This puts the first element in col ',oldx,newx); newx=newx*0.0; newx(i,1)=oldx(i,1); call print('This puts the first element in row ',oldx,newx); newx=newx*0.0; newx( ,i)=oldx( ,i); call print('Whole col copied here',oldx,newx); oldx=rn(matrix(10,5:)); newx= matrix(20,5:); i=integers(4); newx(i,1)=oldx(i,1); call print('This puts the first element in row ',oldx,newx); newx=newx*0.0; newx(i,)=oldx(i,); call print('Whole row copied',oldx,newx); * We subset a matrix here ; a=rn(matrix(10,5:)); call print('Pull off rows 1-3, cols 2-4', a,a(integers(1,3),integers(2,4))); b34srun; 5. Functions or math expressions are not allowed on the left hand side of an equation. Assume the user wants to load another row. The command x(norows(x)+1,)=v; in the sequence x=matrix(3,3:1 2 3 4 5 6 7 8 9); v=vector(3:22 33 44); x(norows(x)+1,)=v; will not work. The correct way to proceed is: x=matrix(3,3:1 2 3 4 5 6 7 8 9); v=vector(3:22 33 44); n=norows(x)+1; x(n,)=v; to produce 1. 2. 3. 4. 5. 6. 7. 8. 9. 22. 33. 44. The command x(i+1)=value; will not work since there is a calculation implicit on the left. The correct code is: j=i+1; x(j)=value; 6. Matrix and array math is supported. If x is a 3 by 3 matrix, the command x=afam(x); will create the 3 by 3 array x. If x is a 3 by 1 array. The command mx=vfam(x); will create a matrix mx containing columns of x. If x=rn(array(3,3:)); vv=vfam(x); vv is a 3 by 3 matrix. To convert x to a vector column by column use vvnew=vector(:x); 7. Keywords should not be used as variable names. If they are the command with this name is "turned off." This can cause unpredictable results with user PROGRAMS, SUBROUTINES and FUNCTIONS. Keywords cannot conflict with user program, subroutine or function names since the users code is not loaded unless a statement of the form call load(name); is given. 8. MATRIX command SUBROUTINES and FUNCTIONS allow passing arguments. For example: call myprog(x,y); y=myfunc(x,y); Character values can be passed and optionally changed. b34sexec matrix; subroutine test(a); call print('In routine test A= ',a); call character(a,'This is a very long string'); return; end; /$ pass in character*8 call test('junk'); call character(jj,'some junk going in'); call print(jj); /$ pass in a character*1 array call test(jj); call print(jj); b34srun; Special characters such as : and | are not allowed in USER SUBROUTINES or FUNCTION calls because of the difficulty of parsing these characters in the user routine. This restriction may change in future versions of the MATRIX command if there is demand. 9. Coding assumptions. Statements such as: y = x-z; are allowed. Statements such as y = -x+z; will not work as intended. The error message will be "Cannot classify sentence Y ...". The command should be given as y = -1.*x + z; or better still y = (-1.*x) + z; A statement y = x**2; where x is real*8 will get a mixed mode message and should be given as y = x**2.; Complex statements such as yhat = b1*dexp(-b2*x)+ b3*dexp(-(x-b4)**2./b5**2.) + b6*dexp(-(x-b7)**2./b8**2.); will not work and should have ( ) around the power expressions and -1.* . yhat = b1*dexp(-1.0*b2*x)+ b3*dexp(-1.0*((x-b4)**2.)/(b5**2.)) + b6*dexp(-1.0*((x-b7)**2.)/(b8**2.)); Examples of MATRIX language statements: The statement y=dsin(x); is an analytic statement that creates the stuctured object y by taking the sin of structured object x. The statements below defines a user subroutine moveaver to calculate a moving average. subroutine moveaver(x,nterms,moveaver); 10. The following code illustrates automatic expansion. x(1)=10.; x(2)=20.; The array x contains elements 10. and 20. Warning: The commands x(1)=10.; x(2)=20; produces an array of 0 20 since the statement x(2)=20 redefines the x array to be integer! This is an easy mistake to make! Computers do what we tell them to do! Statements such as x(0) = 20.; x(-1)= 20.; x(1) = 20.; all set element 1 of x to 20. The x(0) and x(-1) statements will generate a message warning the user. 11. Memory Management Issues. Warning: Automatic expansion inside a subroutine, program, DO loop, dowhile loop, or function can cause the program to "waste" memory since newer copies of the expanded variable will not fit into the old location. The matrix command will have to allocate a new space which will leave a "hole" in memory. B34S provides the capability of the user programming memory management as needed. The command call compress; can be used to compress the workspace in this situation. A better solution is to allocate the variable to the max shape outside the loop. In addition to space requirements, prior allocation will substantially speed up execution. If memory problems are encountered, the command call names(all); can be used to see how the variables are saved in memory and whether as the loop proceeds more space is used. In some situations the max temp number message may be reached if temps cannot be automatically freed. For example compare the following code; n=10; x=array(n:); call names(all); do i=1,n; x(i)=dfloat(i); call names(all); enddo; with n=10; call names(all); do i=1,n; x(i)=dfloat(i); call names(all); enddo; The first job will run faster and not use up memory. This job can be found in the file matrix.mac under MEMORY and should be run by users wanting to write efficient subroutines. The command n=10; call compress(n); allows the user to compress every n times call compress is called. For example: do i=1,nn; * statements here; call compress(100); enddo; Compresses every 100 times the loop calls compress. An alternative is to use the solvefree command as do i=1,2000; call solvefree(:alttemp); * many commands here ; call solvefree(:cleantemp); enddo; Due to enhancements to the memory management in B34S that were part of the 8.10G version, this option only has use if memory management is needed as part of a user model that is called by one of the nonlinear commands. For added detail on this use, see below. In the above example the first call with :alttemp sets %%____ style temp variables in place of the default ##____ style. The command :cleantemp resets the temp style to ##____ and cleans all %%____ temps, leaving the ##_____ style temps in place. If this capability is used carefully, substantial speed gains can be made. In addition the max number of temps will not be reached. Use of this feature slows down processing and is usually not needed. The command call solvefree(:cleantemp2); cleans user temps at or above the current level. This can be useful within a nested call to clean work space. The dowhile loop usually is cycled many times and may need active memory management. An example is: b34sexec matrix; sum=0.0; add=1.; ccount=1.; count=1.; tol=.1e-6; /$ outer dowhile does things 2 times call outstring(2,2,'We sum until we can add nothing!!'); call outstring(2,4,'Tol set as '); call outdouble(20,4,tol); call echooff; call solvefree(:alttemp); dowhile(ccount.ge.1..and.ccount.le.3.); sum=0.0; add=1.; count=1.; dowhile(add.gt.tol); oldsum=sum; sum=oldsum+((1./count)**3.); count=count+1.; call outdouble(2,6,add); add=sum-oldsum; /$ This section cleans temps if(dmod(count,10.).eq.0.)then; call solvefree(:cleantemp); call solvefree(:alttemp); endif; enddowhile; ccount=ccount+1.; call print('Sum was ',sum:); call print('Count was ',count); enddowhile; b34srun; Note: The command call compress; will be ignored if it is used in a program, function or subroutine that is called as part of a nonlinear estimation command such as NLLLSQ, CMAX2 etc. The reason for this restriction is to avoid the possibility of data movement that is not known to the calling command. If memory management is needed in this case, use the solvefree command. Note: The dowhile loop will cycle at least one time since the test is made at the end of the loop. If this is not desired, place the dowhile inside an if statement. 12. Missing data Handling missing data often causes problems. Assume the following code: x=rn(array(10:)); lagx=lag(x,1); y=x-(10.*lagx); goody=goodrow(y); call tabulate(x,lagx,y,goody); Y will contain missing data in row 1. The variable goody will contain 9 non missing value observations. 13. Recursive solutions In many cases the solution to a problem requires recursive evaluation of an expression. While the use of recursive function calls is possible, it is not desirable since there is great overhead in calling the function or subroutine over and over again. The do loop, while still slow, is approximately 100 times faster than the recursive function call. The test problem RECURSIVE in c:\b34slm\matrix.mac documents how slow the recursive function call and do loop are for large problems. Another reason that a recursive function call is not recommended is that the stack must be saved. One way to handle a recursive call is to use the SOLVE statement to define the expression that has to be evaluated one observation at a time. If the expression contains multiple expressions that are the same, a FORMULA can be defined and used in the SOLVE statement. The FORMULA and SOLVE statements evaluate an expression over a range, ONE OBSERVATION AT A TIME. This in contrast to the usual analytic expression which is evaluated completely on the right BEFORE the copy is made. Unlike an expression, a formula or SOLVE statement can refer to itself on the right. The BLOCK keyword determines the order in which the formulas are evaluated. If the expression in the SOLVE statement does not have duplicate code, it is faster not to define a FORMULA. Examples of both approaches are given next. The code: test=array(10:); test(1)=.1; b=.9; solve(test=b*test(t-1)+rn(1.) :range 2 norows(test) :block test); call print(test); works but test = b*lag(test,1)+rn(1.); will not get the "correct" answer since the right hand side is built before the copy is done. More complex expressions. The FORMULA statement requires use of the subscript t unless the variable is a scalar. The use of the FORMULA and SOLVE statements are illustrated below: Example 1. b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; double=array(norows(gasout):); formula double = dlog(gasout(t)*2.); call names; call print(double); test2=array(norows(gasout):); solve(test2=test2(t-1)+double(t)+double(t-1) :range 2, norows(gasout) :block double); call print(mean(test2)); b34srun; Example 2. The following two statements are the same do i=1,n; x(i)=y(i)/2.; enddo; solve(x=y(t)/2. :range 1 n); Note: The SOLVE and FORMULA statements cannot use user functions. A DO loop can use user functions. More detail on the SOLVE and FORMULA statements are given below. An alternative to the FORMULA/SOLVE approach is to call a Fortran or other execuitable directly for the recursive calculation from inside a B34S Matrix command program. For an example of this approach see section 22 below and the examples FORTRAN and FORTRAN2 in the file matrix.mac. This method is very fast and is the best way to go. The down side is building the fortran and having to have a fortran compiler. Since c routines can also be used and there are GPL c compilers, this is not a serious problem, especially on linux where f77 cones with the operating system. 14. User defined data structures The B34S MATRIX command allows users to build custom data types. The below example shows the structure PEOPLE consisting of a name field (PNAME), a SSN field (SSN), an age field (AGE), a race field (RACE) and an income field (INCOME). The built in function SEXTRACT( ) is used to take out a field and the built in SUBROUTINE ISEXTRACT is used to place data in a structure. Both SEXTRACT and ISEXTRACT allow a third argument that operates on an element. The name SEXTRACT is "structure extract" while ISEXTRACT is "inverse structure extract." Use of these commands is illustrated below: b34sexec matrix; people=namelist(pname,ssn,age,race,income); pname =namelist(sue,joan,bob); ssn =array(:20,100,122); age =idint(array(:35,45,58)); race =namelist(hisp,white,black); income=array(:40000,35000,50000); call tabulate(pname,ssn,age,race,income); call print('This prints the age vector',sextract(people(3))); call print('Second person',sextract(people(1),2), sextract(people(3),2)); * make everyone a year older ; nage=age+1; call isextract(people(3),nage); call print(age); * make first person 77 years old; call isextract(people(3),77,1); call print(age); b34srun; Data structures are very powerful and, in the hands of an expert programmer, can be made to bring order to complex problems. Note: If argument(%vnames) is used where ( ) referes to a namelist with variables of 8 characters spaces must be added. This is done by call load(catname :staging); call catname(%vnames,' ',%vnames,0); call makedata(argument(%vnames) :heading 'jtrain2 data' :file 'wood.mac' :member jtrain2); 15. Advanced programming Concepts and Techniques for Large Problems Programs such as Speakeasy which are meant to be used interactively have automatic workspace compression. As a result a Speakeasy LINKULE programmer has to check for movement of defined objects anytime an object is created or freed. So as to increase speed, B34S was designed not to move the variables inside the allocator unless told to do so. If a DO loop terminates and the user is not in a SUBROUTINE, temp variables are freed. If a new temp variable is needed, B34S will try to place this variable in a free slot. If a variable is growing, this may not be possible. Hence it is good programming practice to create arrays and not rely on automatic variable expansion. In a subroutine call, a variable passed in is first copied to another location and set to the current level + 1. Thus there are name storage implications of a subroutine call. In a large program the command call compress; will manually clean out all temp variables and compress the workspace. While this command takes time, in a large job it may be required to save time and space. For example: temp variables are named ##1 ...... ##999999. If the peak number of temp variables gets > 999999, then B34S has to reuse old names and as a result slows down checking to see if a name is currently being used. A call to COMPRESS will reset the temp variable counter as well as free up space. Compacted space in worekingmemory speeds up execution. If call compress is called from a place it cannot run, say in a program subroutine or function that is being called by a b34s command such as cmax2, it will be ignored and no message will be given. The compress command will also be ignored if it is called under a running function, directly or indirectly. This means that a call to compress from a subroutine may or may not compress depending on whether the subroutine was called from a user function. The MATRIX command termination message gives space used, peak space used and peak and current temp # usage. Users can monitor their programs with these measures to optimize performance. In the opinion of the developer, the B34S MATRIX command DO loop is too slow. The problem is that the DO loop will start to run without knowing the ending point because it is supported at the lowest programming level. In contrast, Speakeasy requires that the user have a DO loop only in a SUBROUTINE, PROGRAM or FUNCTION where the loop end is known in theory. Ways to increase DO loop speed are high on the "to do" list. Faster CPU's may be the answer. The LF95 compiler appears to make faster DO loops than the older LF90 compiler. This suggests that the cache may be part of what is slowing things down. The test problem SOLVE6 illustrates some of these issues. Times and gains from the SOLVE statement vary based on the compiler used to build the B34S. On a Gateway P6 400 machine LF90 4.50i LF95 5.5b SOLVE time 9.718 9.22 DO time 41.69 13.73 Gain of SOLVE 4.3897 1.49 On a Dell Latitude 1MH machine using verions 8.10X the SOLVE and DO times were 2.6538162 and 3.8855877 respectively giving a gain of 1.4641510. LF95 7.10 was used to build b34s. LF90 appears to make a very slow DO loop!! LF95 is faster. In simple formulas the FORMULA and SOLVE commands are useful. With large complex sequences of commands, the DO loop cost may have to be "eaten" by the user since it is relative low in comparison to the cost of parsing the expression. A major advantage of the DO loop is that the logic is 100% clear is most cases. Speed can be increased by using variables for constants. This is because at parse time all scalars are made temps. Doing this outside the loop speeds things. Slow code do i=3,1000; x(i)=x(i)*2.; enddo; better code two=2.0; do i=3,1000; x(i)=x(i)*two; enddo; Vectorized code i=integers(3,1000); x=x(i)*2.; Compact vectorized code x=x(integers(3,1000))*2.; If all elements need to be changed the fastest code is x=x*2.; In the vectorized examples parse time is the same no matter whether there are 10 elements in x or 10,000,000. For speed gains from the use of masks, see # 18 below. Since B34S can create, compile and execute Fortran programs, for complex calculations a branch to Fortran is always an option. The larger the dataset the less the overhead costs. The Fortran example in matrix.mac illustrates dynamically building, compiling and calling a Fortran program from the matrix command. b34sexec matrix; call open(70,'_test.f'); call rewind(70); /$ 1234567890 call character(test," write(6,*)'This is a test # 2'" " n=1000 " " write(6,*)n " " do i=1,n " " write(6,*) sin(float(i)) " " enddo " " stop " " end "); call write(test,70); call close(70); /$ lf95 is Lahey Compiler /$ g77 is Linux Compiler /$ fortcl is script to run Lahey LF95 on Unix to link libs call dodos('lf95 _test.f'); * call dounix('g77 _test.f -o_test'); call dounix('lf95 _test.f -o_test'); * call dounix('fortcl _test.f -o_test'); call dodos('_test > testout':); call dounix('./_test > testout':); call open(71,'testout'); call character(test2, ' '); call read(test2,71); call print(test2); testd=0.0; n=0; call read(n,71); testd=array(n:); call read(testd,71); call print(testd); call close(71); call dodos('erase testout'); call dodos('erase _test.f'); call dounix('rm testout'); call dounix('rm _test.f'); b34srun; For a more advanced example, see FORTRAN_2 test case in matrix.mac. 16. DO, IF and DOWHILE nested statement limits Do loop, DOWHILE loop and IF statement termination should be hit. If this is not done, the max IF statement limit or DO statement limit can be exceeded depending on program logic. This "limitation" comes from having IF and DO loops outside programs, subroutines or functions. When the DO loop or IF structure starts to run the program does not know the end. The code loop continue; if(dabs(z1-z2).gt.1.d-13)then; z2=z1; z1=dlog(z1)+c; go to loop; endif; will never hit endif; The b34s parser will not know the position of this statement and the max IF statement limit could be hit if the IF structure was parsed many times. A better approach is not to use an IF structure in this situation. The correct code is: loop continue; if(dabs(z1-z2).le.1.d-13)go to nextstep; z2=z1; z1=dlog(z1)+c; go to loop; nextstep continue; 17. Mask Issues Assume an array x where for x < 0 we want y=x**2. while for x GE 0.0 we want y=2*x. A slow way to do this would be: do i=1,norows(x); if(x(i) .lt. 0.0)y(i)=x(i)**2.; if(x(i) .ge. 0.0)y(i)=x(i)*2. ; enddo; since the larger the X array the more parsing is required since the do loop cycles more times. A vectorized way to do the same calculation is to define two masks. Mask1 = 0.0 if the logical expression is false, = 1.0 if it is true. Fast code would be mask1= x .lt. 0.0 ; mask2= x .ge. 0.0 ; y= mask1*(x**2.0) + mask2*(x*2.0); Faster code would be y= (x .lt. 0.0)*(x**2.0) + (x .ge. 0.0 )*(x*2.0); Complete problem: b34sexec matrix; call print('If X GE 0.0 y=20*x. Else y=2*x':); x=rn(array(20:)); mask1= x .lt. 0.0 ; mask2= x .ge. 0.0 ; y= mask1*(x*2.0) + mask2*(x*20.); call tabulate(x,y,mask1,mask2); b34srun; Compact code (placing the logical calculation in expression) is: b34sexec matrix; call print('If X GE 0.0 y=20*x. Else y=2*x':); x=rn(array(20:)); y= (x.lt.0.0)*(x*2.0) + (x.ge.0.0)*(x*20.); call tabulate(x,y); b34srun; Logical mask expressions can be used in function and subroutine calls to speed calculation. 18. N Dimensional Objects While the Matrix command saves only 1 and 2 dimensional objects, it is possible to save and address n dimensional objects in 1 d arrays. n dimensional objects are saved by col. The commands: nn=index(i,j,k:); x=array(nn); call setndimv(index(i,j,k),index(1,2,3),x,value); will make an 3 dimensional(i,j,k) object x and place value in the 1 2 3 position. The function call yy=getndimv(index(i,j,k),index(1,2,3),x); or yy=x(index(i,j,k:1,2,3)); can be used to pull a value out. For example to define the 4 dimensional object x with dimensions 2 3 4 5 nn=index(2,3,4,5:); x=array(nn:); To fill this array with values 1.,...,norows(x) x=dfloat(integers(norows(x))); to set the 1 2 3 1 value to 100. call setndim(index(2,3,4,5),index(1,2,3,1),x,100.); Examples: b34sexec matrix; x=rn(array(index(4,4,4:):)); call print(x,getndimv(index(4,4,4),index(1,2,1),x)); do k=1,4; do i=1,4; do j=1,4; test=getndimv(index(4,4,4),index(i,j,k),x); call print(i,j,k,test); enddo; enddo; enddo; b34srun; b34sexec matrix; xx=index(1,2,3,4,5,4,3); call names(all); call print(xx); call print('Integer*4 Array ',index(1 2 3 4 5 4 3)); call print('# elements in 1 2 3 4 is 24',index(2 3 4:)); call print('Position of 1 2 in a 4 by 4 is 5',index(4 4:1 2):); call print('Integer*4 Array ',index(1,2,3,4,5 4 3)); call print('# elements in 1 2 3 5 is 30',index(2,3,5:)); call print('Position of 1 3 in a 4 by 4 is 9',index(4,4:1,3):); b34srun; b34sexec matrix; mm=index(4,5,6:); xx=rn(array(mm:)); idim =index(4,5,6); idim2=index(2,2,2); call setndimv(idim,idim2,xx,10.); vv= getndimv(idim,idim2 ,xx); call print(xx,vv); b34srun; N dimensional arrays are very good at saving data. What thgey are not good is directly processing the data. Here we need to extract what is needed into 1D or 2D objects, make the needed calculations and replace the values back in the structure. 19. Complex Math Issues If x=complex(1.5,1.5); y=complex(1.0,0.0); a=x*y; produces a=(1.5,1.5); to zero out imag part of a use a=complex(real(x*y),0.0); Assume x is a real*8 array and y=2., the statement newx=x**y; squares all elements. If y is not equal to a positive integer greater than 1. then x must be made complex prior to the calculation. Now assume x is a matrix. If y is a positive integer, then x can be real. If y is not an integer, then both x and y must be complex*16. The calculation proceeds as follows. First do a eigenvalue factorization of x as x = c * dlamda * inv(c) where c is the eigenvectors of x and dlamda is a diagonal matrix with eigenvalues along the diagonal. We note that x**y = c * (dlamda**y) * inv(c) which has transformed what would have been a very difficult calculation into an element by element operation along the diagonal. RG option: To speed up calculation x is inspected. If the imag part is all 0.0, the EISPACK RG routine was used. This is substantially faster than the EISPACK CG which is used when x is a complex number. The RG calculation seems to differ in accuracy from MATLAB. It appears that MATLAB is using a complex eigenvalue analyis to form the power. To make B34S run simular to MATLAB, using RG is no longer the default. If the user wants to force RG to be used in these situations where x is real place line b34sexec options debugsubs(b34smatpower2); b34srun; prior to calling b34sexec matrix; 20. Inversion Issues. Assume X is a n by n real*8, real*16 or complex*16 or complex*32 matrix or VPA matrix. The command invx=inv(x); will invert x using the LINPACK LU factorization routines. If x is not full rank, the progam will stop with an error message. The optional argument rcond in invx=inv(x,rcond); will return the condition. If it is known that x is symmetric or positive definate, the key words :smat or :pdmat can be used to speed up the calculation 3 or more times. The default is the LU factorization using LINPACK routine DGECO/DGEDI and ZGECO/ZGEDI. The optional argument :gmat will force use of the LAPACK routines DGETRF/ZGETRF and DGETRI/ZGETRI. The switch :pdmat2 will call the LAPACK routines DPOTRF/DPOTRI in place of LINPACK. LAPACK is not available for real*16, complex*32 or VPA matrices. For example invx=inv(x); invx=inv(x:gmat); invx=inv(x:smat); invx=inv(x:pdmat); invx=inv(x:pdmat2); A number of jobs can be run to illustrate the speed gains. default uses LINPACK DGECO/DGEDI and ZGECO/ZGEDI gmat uses LAPACK DGETRF/ZGETRF and DGETRI/ZGETRI smat uses LINPACK DSICO/DSIDI and ZSICO/ZSIDI pdmat uses LINPACK DPOCO/DPODI and ZPOCO/ZPODI pdmat2 uses LAPACK DPOTRF/DPOTRI and ZPOTRF/ZPOTRI The LAPACK library has routines DGESVX and ZGESVX that allows solutions of the system X*A=B where x is n by n, a is n by m and b is n by m using refinement and or balancing of the system. While it is not space efficient, if b is set as the identity matrix, such routines can be used to refine the inverse. The optional key words :refine and :refinee allow solution of the inverse where the solution is refined and or refined and equilibrated. invx=inv(x:refine); invx=inv(x:refinee); The LAPACK rcond estimator differs slightly from the LINPACK rcond estimator. For a symmetric positive definate matrix, the Cholesky routines r=pdfac(x); xinv=pdinv(r); can be used. Rank problems can be detected with optional arguments. r=pdfac(x,rcond); r=pdfac(x,rcond,ibad); and solutions of equations can be calculated with a=pdsolv(r,b); The Cholesky r can be updated and down dated using the commands pdfacdd and pdfacud. The commands pdfac, pdinv, pdsolv, pdfacdd and pdfacud work with real*4, real*8, real*16, complex*16 and complex*32 matrixes. While the above solvers all use LINPACK, it is well known that the QR method can be used to obtain r. Assuming x is n by k, the command r1 = qrfac(x); produces the same r as r2=pdfac(transpose(x)*x); Using the QR routines LINPACK DQRDC and ZQRDC routines, and their real*16 and complex*32 counterparts qqrdc and cqqrdc, it is possible to solve the OLS problem as: qr=qrfac(x,pivot); b=qrsolve(qr,pivot,y,info); where qrsolve uses the LINPACK DQRSL / ZQRSL or QQRSL and CQQRSL routines QRFAC and QRSOLV work for real*4,real*8, real*16, complex*16 or complex*32 data. Real and complex VPA (variable precision arimatic) For a real*8 matrix, the generalized inverse can be obtained by ginv=pinv(x,irank); ginv=pinv(x,irank,toll); where: irank = an estimate of the rank of x toll = tolerance that is used to set the singular values to zero. At present pinv works for a real*8 matrix x. IMSL routine DLSGRR is used for the calculation. In many applications one may want to determine easily if a matrix is full rank and take corrective action without placing the inverse in the code. The call call gmfac(x,l,u,info); Factors n by n matrix x such that x = L*U where L is lower triangular with 1.0 on the diagonal and U is upper triangular. In contrast to x=inv(xx); which uses LINPAC DGECO/DGEFA/DGEDI ZGECO/ZGEFA/ZGEDI, and for real*16 and complex*32 QGECO/QGEFA/QGEDI CQGECO/CQGEFA/CQGEDI GMFAC uses the LAPACK routines DGETRF and ZGETRF. GMFAC optionally will return info > 0 if U(i,i)=0. GMFAC will not run on real*16 and complex*32 data types. The commands call gminv(x,xinv); call gminv(x,xinv,info); call gminv(x,xinv,info,rcond); invert a general matrix using LAPACK. This code may be faster than inv( ) if the rank of the matrix is greater than 200 by 200. If the optional argument info is present, the routine will not stop if there is a problem. This allows the user to take automatic corrective action. The routine GMINV is the fastest large general matrix routine. It does not do refinement by default. The rcond option takes time and usually is not needed. The rcond value from LAPACK is not the same as LINPACK. The routine GMINV uses LAPACK DGETRF/ZGETRF and DGETRI/ZGETRI and optionally DGECON/DGECON if the condition is needed. The command call gmsolv(x,b,ans,info); where x is a n by n matrix, b a n by k matrix solves the system ans*x=b. If the optional argument info is present, the program will not stop if there is a rank problem. The LAPACK routines DGETRF/DGETRS and ZGETRF/ZGETRS are usually used. If the key words :refine or :refinee are present, the LAPACK routines DGESVX and ZGESVX will be used to refine the solution. This will take substantially more time. Unless the optional key words are used, gmsolv is the fastest way to solve the general system ans*x=b where x is greater than or equal to 200 by 200. The larger b, the more the gain in speed. The command s=svd(x,ibad,job,u,v); Calculates singular value decomposition of x which must be real*8 or real*16, complex*16 or complex*32. LINPACK routines DSVDC and ZSVDC are used for real*8 or complex*16 x. For real*16 and complex*32, these have been converted to QSVDC and CQSVDC respectively. For added detail on these options, please look at the help files and the example files. Inversion options under the matrix command have been designed to be both easy to use and flexible. If the series are real*16 or complex*32, only the LINPACK routines are available since LAPACK is not currently available for these data types. The inv( ) command also works for VPA data. Substantial accuracy gains can be obtained. For further detail, see sections 23 & 24 below. 21. Fortran and other Language Interfaces While the B34S Matrix language is "vectorized" and can easily handle most tasks, recursive systems that involve DO and DOWHILE loops are quite slow. An alternative is to code the desired calculation in the user language of choice and link to this program inside a B34S matrix command subroutine. While one possible implementation of this facility might be via a W2K DLL, this approach was rejected as being not portable and overly complex. The implementation discussed below allows the user to use any language to develop the program and communicate with the b34s processor with files. The below listed examples illustrate this technique. If there is sufficient user demand, at a later point a user DLL for the matrix command might be developed. The downside is that such an implementation would require the user directly access the matrix commmand named storage array. It is the view of the developer of B34S that only the most experienced programmers are capable of this complexity. A programmer without the proper training could easily "kill" or worse still, damage the B34S MATRIX command resulting in a hard-to-detect error being generated. The file IO approach will not allow direct access to the B34S matrix command arrays and is safer but slower. If most of the cpu time is in the calculation, not in passing the data, this cost will be minimal. Example of a user fortran program being called. The below listed example calls the program _test.exe on W2K which in turn writes a file of 1000 sin values. These are read into the matrix command. Except for the compile command this job is 100% portable between Linux and W2K. The user is NOT required to write in FORTRAN. Any user programming language or user external program is allowed provided it can read and make files for the IO into the B34S matrix command. Simple Example Program logic: 1. Open file _test.f 2. Copy lines in the 2-D character*1 array test into this file. 3. Compile the fortran. 4. Run the program _test and place output in testout 5. Open testout in the B34S matrix command and read the character first line. The number of elements line into valiable n. Allocate a 1d array testd of length n. Read data into testd. Print testd. b34sexec matrix; call open(70,'_test.f'); call rewind(70); /$ 1234567890 call character(test," write(6,*)'This is a test # 2'" " n=1000 " " write(6,*)n " " do i=1,n " " write(6,*) sin(float(i)) " " enddo " " stop " " end "); call write(test,70); call close(70); call dodos('lf95 _test.f'); call dounix('lf95 _test.f -o_test'); call dodos('_test > testout':); call dounix('./_test > testout':); call open(71,'testout'); call character(test2,' '); call read(test2,71); call print(test2); testd=0.0; n=0; call read(n,71); testd=array(n:); call read(testd,71); call print(testd); call close(71); call dodos('erase testout'); call dodos('erase _test.f'); call dounix('rm testout'); call dounix('rm _test.f'); b34srun; Complex Example Program Logic 1. Fortran program logic: Read data from data.dat into array data1. Read number of parameters in the model into npar. Read Model npar parameters into array parm. Calculate the function value and write to file testout. 2. Matrix program test writes the parameters as they change to a file and calls the compiled fortran program which will return with a new func value. Things to be aware of: The commands call dodos(' '); call dounit(' '); were not given in the form call dodos(' ':); call dounit(' ':); since there is no screen writting. By not using the : the flash is avoided. The matrix command write used for data was call write(y,72,'(3e25.16)'); in place of the more general call write(y,72); because the line got too long on Linux. The lines call out.... were added to show progress. These are only seen if the program is running in the Display Manager. b34sexec options ginclude('b34sdata.mac') member(lee4); b34srun; b34sexec matrix ; call loaddata ; * The data has been generated by GAUSS by with settings $ * a1 = GMA = 0.09 $ * b1_n = GAR = 0.5 ( When Negative) $ * b1 = GAR = 0.01 $ * call echooff ; /$ Setup fortran call open(70,'_test.f'); call rewind(70); call character(fortran, /$234567890 " implicit real*8(a-h,o-z) " " parameter(nn=10000) " " dimension data1(nn) " " dimension res1(nn) " " dimension res2(nn) " " dimension parm(100) " " call dcopy(nn,0.0d+00,0,data1,1)" " call dcopy(nn,0.0d+00,0,res2 ,1)" " open(unit=8,file='data.dat') " " open(unit=9,file='tdata.dat') " " read(8,*)nob " " read(8,*)(data1(ii),ii=1,nob) " " read(9,*)npar " " read(9,*)(parm(ii),ii=1,npar) " " read(9,*) res2(1) " " close(unit=9) " " " " do i=1,nob " " res1(i)=data1(i)-parm(3) " " enddo " " " " func=0.0d+00 " " do i=2,nob " " res2(i) =parm(1)+(parm(2)* res2(i-1) ) +" " * (parm(4)*(res1(i-1)**2) ) " " func=func+(dlog(dabs(res2(i))))+ " " * ((res1(i)**2)/res2(i)) " " enddo " " func=-.5d+00*func " " close(unit=8) " " open(unit=8,file='testout') " " write(8,fmt='(e25.16)')func " " close(unit=8) " " stop " " end "); call write(fortran,70); call close(70); maxlag=0 ; y=doo1 ; y=y-mean(y) ; * compile fortran and save data; call dodos('lf95 _test.f' ); * call dounix('lf95 _test.f -o_test'); call dounix('fortlc _test'); call open(72,'data.dat'); call rewind(72); call write(norows(y),72); call write(y,72,'(3e25.16)'); call close(72); v=variance(y) ; arch=array(norows(y):) + dsqrt(v); i=2; j=norows(y); * parm=array(:.0001 .0001 .0001 .0001); * parm(1)=v; * parm(3)=mean(y); count=0.0; call echooff; program test; call open(72,'tdata.dat'); call rewind(72); npar=4; call write(npar,72); call write(parm,72,'(e25.16)'); arch(1)=0.0d+00 ; call write(arch(1),72,'(e25.16)'); call close(72); call dodos('_test'); call dounix('./_test '); call open(71,'testout'); func=0.0; call read(func,71); call close(71); count=count+1.0; call outdouble(10,5 ,func); call outdouble(10,6 ,count); call outdouble(10,7, parm(1)); call outdouble(10,8, parm(2)); call outdouble(10,9, parm(3)); call outdouble(10,10,parm(4)); return; end; ll =array(4: -.1e+10, .1e-10,.1e-10,.1e-10); uu =array(4: .1e+10, .1e+10,.1e+10,.1e+10); rvec=array(4: .1 .1, .1, .1); parm=rvec; * call names(all); call cmaxf2(func :name test :parms parm :ivalue rvec :maxit 2000 :maxfun 2000 :maxg 2000 :lower ll :upper uu :print); *call dodos('erase testout'); *call dodos('erase _test.exe'); *call dounix('rm testout'); *call dounix('rm _test'); b34srun; 22. Polynomial Matrix Operations A polynomial matrix is one where the coeffiients are themselves coefficients. A two by two system with max order 3 could be saved in form B(1,1)(L) B(1,2)(L) B(2,1)(L) B(2,2)(L) where the zero order terms (constant) are saved in the term. The VAREST command saves the matrix in a form where the constant is placed last. This is NOT :BYVAR format which would have occured if call olsq(y y{1 to maxlag} x{1 to maxlag} were used. B(1,1)(L) B(1,2)(L) c(1) B(2,1)(L) B(2,2)(L) c(2) For a two variable system with 6 lags the coefficients are saved in a 2 by ((2*6)+1) matrix. VAREST also saves [I - B(L)] which when inverted gives the phi weights. The matrix polynomial routines POLYMINV (used to calculate the inverse of a matrix), POLYMMULT (used to calculate a multiplication) and POLYMDISP (used to print or display such a matrix) use the convention: B(1,1)(0) B(1,2)(0) B(1,1)(k) B(1,2)(k) B(2,1)(0) B(2,2)(0) B(2,1)(k) B(2,2)(k) which is termed :BYORDER The command POLYMCONV can be used to convert from one system to another. Each matrix in :BYORDER form has a three element integer variable of the form (nrow,ncol,degree+1) that allows it to be decoded. If nrow=ncol then the matrix can be inverted. A matrix in :BYVAR form does not need an index to decode it but must be saved as a n by ((n*order)+1) system. For example a 2 variable VAR with mar order 5 could be saved as a 2 by 11. The first row would be 5 lags of x(1) and 5 lags x(2) plus the constant. For further detail see the VAREST command example. A simple setup is: b34sexec matrix; call loaddata; call load(buildlag); call load(varest); call echooff; ibegin=1; iend=296; nlag=2; nterms=10; x=catcol(gasin,gasout); call varest(x,nlag,ibegin,iend,beta,t,sigma,corr,residual,1, a,ai,varx,varxhat,rsq); call print(beta,t,sigma,corr); call tabulate(varx,varxhat,rsq); call polymdisp(:display a ai); call polyminv(a ai psi ipsi nterms); call polymdisp(:display psi ipsi); b34srun; 23. Real*16 and Complex*32 Data types While the usual data types in the MATRIX command are real*8 and complex*16, there is a limited real*16 and complex*32 capability. In addition there are improvements to accuracy in real*8, real*16, complex*16 and complex*32 by modification the the BLAS routines. If higher accuracy is needed, the VPA option, discussed in the next section, can be used. Examples: Option 1. Works for all commands that use routines that involve BLAS routines. The command call real16on(:real16math); will perform math with real*8 objects using real*16 math and complex*16 objects using complex*32 math. The command call real16on; uses the IMSL routines dqadd and dqmult to increase accuracy. It is faster than call real16on(:real16math). These options can be turned off with the command call real16off; For example the default test case takes 2.04, with real16on it takes 3.14 while, with real16on(:real16math) set, the time is 5.95. In most cases this added accuracy is not needed. For real*16 and complex*32 objects, the command call real32on; uses added accuracy for qdot, qsum, qasum, qaxpy and qscal and added accuracy for cqdot, cqsum, cqasom, cqaxpy and cqscal call real32off; turns off this feature. The command call real32_vpa; uses vpa math to increase real*16 accuracy. This will run slowly but variable precision math is supported. This allows vpa math to support a number of otherwise real*16 commands. At a later date this may enhance complex*32 objects. For now it is experimental. Option 2. Real*16 and complex*32 variables can be created from real*8 and complex*16 variables with the r8tor16 and c16toc32 commands. The commands r16tor8 and c32to16 can be used to convert series back to the default datatypes. A series can be created to be a specific data type with the command kindas( ). Assume x is real*8 and xr16 is real*16. The user wants one_r8 to be real*8 and oner_16 to be real*16. The following commands can be used. one_r8=kindas(x,1.0); oner_16=kindas(xr16,1.0); A more accurate way to proceed is oner_16=kindas(xr16,real16('1.0')); oner_r8=kindas(x, real16('1.0')); While array and matrix math is 100% available, only a subset of commands are available for real*16 and complex*32. More detail is available under the help commands of the specific commands. See the matrix example r16c32 and the examples math5 and math6 for applications. As of 24 June 2003, INV, EIG, SEIG, SVD and a number of production commands such as DEXP, DLOG, SUMSQ, SUM etc have been converted. While LINPACK, and EISPACK have been converted to support real*16 and complex*32, FFTPACK and LAPACK have not. Warning: While real*8 and compolex*16 data can be converted to real*16 and complex*32 with the commands r8tor16( ) and c16toc32 respectively, much accuracy will be lost from the initial data read into real*8 and complex*16. For more accuracy it is it necessary to read into real*16 or complex*32 directly. For an example, of how this is done see the Filippelli (filippelli.dat) data which is in stattest.mac. 24. Variable Precision Arithmetic. The B34S has the capability of variable precision math where the number of digits calculated is up to 1780. For more detail on this facility see the help documents for subroutine VPASET and function VPA that provide the interface into this facility. Full matrix and array math is provided as well as the inverse or real and complex VPA objects. Unlike a number of other software systems, this facility is 100% integrated in the b34s system. For example, once a vpa number is built with the vpa(function), calculations can be made in the usual manner as will be illustrated below. Real, integer and complex objects have been implemented. The facility allows hugh numbers, both real and integer, to be used in calculations. Given the default settings of MBASE is 10**7 and NDIGMX = 256, integers less than 10**1792 can be used. In contrast, the range for integer*4 is -2,147,483,648 - 2,147,483,647. For integer*8 these become -9,223,372,036,854,775,808 - 9,223,372,036,854,775,807. This facility was built with the FM_ZM Library version 1.2 built by David M. Smith. The basic reference is: Algorithm 693, ACM Transactions on Mathematical Software, Vol. 17, No. 2, June 1991, pages 273-283. although a newer version of the libraty was obtained from the web. The below listed program calculates 2.0/4.11 a number of ways and shows the various accuracy values obtained. b34sexec matrix; call echooff; * Accuracy differences depending on data precision; call print(' 2.0/4.11 using different precisions':); call fprint(:clear :col 32 :string ' 10 20 30 40 50' :print :clear :col 32 :string '12345678901234567890123456789012345678901234567890' :print); call fprint(:clear :col 1 :string 'str => vpa' :col 30 :display vpa('2. ')/vpa('4.11') 'e70.52' :print :clear :col 30 :display vpa('2.00')/vpa('4.11') 'e50.32' :print); call fprint(:clear :col 1 :string 'real*8 => vpa' :col 30 :display vpa(2.00)/vpa(4.11) 'e50.32' :print); call fprint(:clear :col 1 :string 'real*8 => real*16' :col 18 :display r8tor16(2.)/r8tor16(4.11) '(e50.32)' :print); call fprint(:clear :col 1 :string 'str => real*16' :col 18 :display real16('2.00')/real16('4.11') '(e50.32)' :print); call fprint(:clear :col 1 :string 'real*8 => real*8' :col 18 :display array(:2.00/4.11) '(e50.32)' :print); call fprint(:clear :col 1 :string 'real*4 => real*4' :col 18 :display sngl(2.00)/sngl(4.11) '(e50.32)' :print); b34srun; The Above job shows 2./4.11 using various precisions. The VPA result to 52 digits is: .4866180048661800486618004866180048661800486618004866M+0 and shows the repeating pattern that will continue. When the data (2.0 and 4.11) was read with a string input, no accuracy was lost with real*16 given the format e50.32. This degree of accuracy was not obtained with real*16 if the data was read as real*8 first and then converted to real*16. The above job was edited to display only 72 columns and is listed below. 2.0/4.11 using different precisions 10 20 30 40 1234567890123456789012345678901234567890 str => vpa .4866180048661800486618004866180048661800 .48661800486618004866180048661800M+0 real*8 => vpa .48661800486618001080454992664677M+0 real*8 => real*16 0.48661800486618001080454992664677E+00 str => real*16 0.48661800486618004866180048661800E+00 real*8 => real*8 0.48661800486618000000000000000000E+00 real*4 => real*4 0.48661798200000000000000000000000E+00 Even for this simple case, the real*4 result began to differ at the 6th digit (although with rounding the failure was at 10). The real*8 result failed at digit 17 as expected. When the real*16 data input was by string there was no loss of accuracy up to the 32nd digit. However, when the data was first read as real*8 and converted to real*16 accuracy problems emerged by the 17th digit. The calculation was limited by the precision of the data read. This finding was verified when we moved real*8 data into VPA and got the same result as the real*8 into real*16. The str into vpa is the "true" answer. The next example show inverse gains: /; /; Shows gains in accuracy of the inverse with vpa /; b34sexec matrix; call echooff; n=3; call vpaset(:ndigits 1750); x=rn(matrix(n,n:)); r16x=r8tor16(x); vpax=vpa(x); call print('Real*4 tests',sngl(x),inv(sngl(x)),sngl(x)*inv(sngl(x))); call print('Real*8 tests',x, inv(x), x*inv(x)); call print('Real*16 tests',r16x,inv(r16x),r16x*inv(r16x)); call print('VPA tests',vpax,inv(vpax),vpax*inv(vpax)); b34srun; Edited output produces: Real*4 tests Matrix of 3 by 3 elements (real*4) 1 2 3 1 2.05157 1.27773 -1.32010 2 1.08325 -1.22596 -1.52445 3 0.825589E-01 0.338526 -0.459242 Matrix of 3 by 3 elements (real*4) 1 2 3 1 0.521061 0.675528E-01 -1.72205 2 0.179445 -0.402323 0.819689 3 0.225948 -0.284424 -1.88285 Matrix of 3 by 3 elements (real*4) 1 2 3 1 1.00000 0.919681E-08 0.759197E-07 2 0.585772E-07 1.00000 0.822609E-07 3 0.878866E-08 0.560976E-08 1.00000 Real*8 tests X = Matrix of 3 by 3 elements 1 2 3 1 2.05157 1.27773 -1.32010 2 1.08325 -1.22596 -1.52445 3 0.825589E-01 0.338525 -0.459242 Matrix of 3 by 3 elements 1 2 3 1 0.521061 0.675528E-01 -1.72204 2 0.179445 -0.402323 0.819689 3 0.225948 -0.284424 -1.88285 Matrix of 3 by 3 elements 1 2 3 1 1.00000 -0.555112E-16 0.00000 2 0.00000 1.00000 0.00000 3 -0.138778E-16 -0.277556E-16 1.00000 Real*16 tests R16X = Matrix of 3 by 3 elements (real*16) 1 2 3 1 2.05157 1.27773 -1.32010 2 1.08325 -1.22596 -1.52445 3 0.825589E-01 0.338525 -0.459242 Matrix of 3 by 3 elements (real*16) 1 2 3 1 0.521061 0.675528E-01 -1.72204 2 0.179445 -0.402323 0.819689 3 0.225948 -0.284424 -1.88285 Matrix of 3 by 3 elements (real*16) 1 2 3 1 1.00000 -0.144445E-33 0.00000 2 0.00000 1.00000 0.00000 3 0.00000 -0.240741E-34 1.00000 VPA tests VPAX = Matrix of 3 by 3 elements VPA - FM 1 2 3 1 .205157M+1 .127773M+1 -.132010M+1 2 .108325M+1 -.122596M+1 -.152445M+1 3 .825589M-1 .338525M+0 -.459242M+0 Matrix of 3 by 3 elements VPA - FM 1 2 3 1 .521061M+0 .675528M-1 -.172205M+1 2 .179445M+0 -.402323M+0 .819689M+0 3 .225948M+0 -.284424M+0 -.188285M+1 Matrix of 3 by 3 elements VPA - FM 1 2 3 1 .100000M+1 -.139488M-1757 .000000M 0 2 .168219M-1758 .100000M+1 .000000M 0 3 -.138187M-1758 .173950M-1758 .100000M+1 and shows the worse accuracy of the off diagonal of the real*4 case was 0.822609E-07 while the worse VPA error was .168219e-1758. The problem consists of random numbers and is designed to be easy. There can be gains is the data is read intro character then into higher precision. the job VPA8 in matrix,mac shows this and provides a template for character data reading: VPA8 Very Hard OLS Data. /; /; Problem very very hard!!! /; Illustrates reading into real*8 and then into real*16 and VPA /; Reading from character into real*16 /; Reading from chapter into VPA /; b34sexec data heading('Bruce McCullough Test Case'); input y x; datacards; 10000000001 1000000000.000 10000000002 1000000000.000 10000000003 1000000000.900 10000000004 1000000001.100 10000000005 1000000001.010 10000000006 1000000000.990 10000000007 1000000001.100 10000000008 1000000000.999 10000000009 1000000000.000 10000000010 1000000000.001 b34sreturn; b34srun; b34sexec matrix; call loaddata; call echooff; /; /; Data reread into character to see what happens /; datacards; 10000000001 1000000000.000 10000000002 1000000000.000 10000000003 1000000000.900 10000000004 1000000001.100 10000000005 1000000001.010 10000000006 1000000000.990 10000000007 1000000001.100 10000000008 1000000000.999 10000000009 1000000000.000 10000000010 1000000000.001 b34sreturn; y16=r8tor16(y); x16=r8tor16(x); /; Cholesky on real*8 fails /; call olsq(y x :print); /; QR fails /; call olsq(y x :qr :print); /; real*16 Cholesky fails /; call olsq(y16 x16 :print); call olsq(y16 x16 :qr :print :savex); coef8_16=rollright(%coef); /; Go to vpa 64 digits and 20 digits to see if makes a difference y_vpa=vpa(%y); x_vpa=vpa(%x); beta_vpa=inv(transpose(x_vpa)*x_vpa)*transpose(x_vpa)*y_vpa; call print('Beta from VPA - default 64 digits accuracy ',beta_vpa); call print(beta_vpa(1)); call print(beta_vpa(2)); call vpaset(:ndigits 200); y_vpa=vpa(%y); x_vpa=vpa(%x); beta_vpa=inv(transpose(x_vpa)*x_vpa)*transpose(x_vpa)*y_vpa; call print('Beta from VPA - 200 digits accuracy ',beta_vpa); call print(beta_vpa(1)); call print(beta_vpa(2)); /; Reading data as character into VPA and real*16 /; Makes a difference !!!!!!!!!!!!!!!!!!!!!!!!!!! data16=r8tor16(array(20:)); call read(data16,4); call load(ntokin :staging); call load(getr16 :staging); call load(getvpa :staging); call load(getchar :staging); call vpaset(:ndigits 200); vpdata=vpa(data16); vpx=vpdata*vpa('0.0'); call rewind(4); cc=c1array(10000:); call read(cc,4); call ntokin(cc,nfind,0,ibad); call print(nfind ); call getvpa(cc,nfind,vpx,ibad); call free(cc); xm=matrix(10,2:data16); y16=array(:xm(,1)); x16=matrix(10,2:data16); x16(,1)=real16('1.'); /; This will not solve due to condition /; call olsq(y16 x16 :print :noint); /; call print('Here data read into real*16 directly - Note difference':); call olsq(y16 x16 :qr :print :noint); call print(' ':); call print('Display More digits to see what real*16 has':); call print(' ':); call fprint(:clear :display %coef(1) '(1pe48.32)' :print); call fprint(:clear :display %coef(2) '(1pe48.32)' :print); call print(' ':); call print('Compare with reading into real*8 then to real*16':); call print(' ':); call fprint(:clear :display coef8_16(1) '(1pe48.32)' :print); call fprint(:clear :display coef8_16(2) '(1pe48.32)' :print); call print(' ':); /; now run with vpa vpxm=matrix(10,2:vpx); vpy= vector(:vpxm(,1)); vpx=vpxm; vpx(,1)=vpa('1.0'); betavpa=inv(transpose(vpx)*vpx)*transpose(vpx)*vpy; call print(' ':); call print('Beta from VPA where character read into VPA':); call print('This is the right way to attack the problem':); call print('For gain compare with real*16 where character read':); call print(betavpa(1)); call print(betavpa(2)); call print(' ':); call print('Beta from VPA where vpa converted from real*8':); beta_vpa=rollright(beta_vpa);; call print(beta_vpa(1)); call print(beta_vpa(2)); call print(' ':); call print('Difference of coef # 1 and # 2':); call print('Shows effect of not reading in correct precision':); diffcoef=vpa(vector(2:)); diffcoef(1)=betavpa(1)-beta_vpa(1); diffcoef(2)=betavpa(2)-beta_vpa(2); call print(diffcoef(1)); call print(diffcoef(2)); b34srun; Heavily Edited output is: Variable # Cases Mean Std Deviation Variance Y 1 10 0.1000000001E+11 3.027650354 9.166666667 X 2 10 1000000001. 0.5278048818 0.2785779932 CONSTANT 3 10 1.000000000 0.000000000 0.000000000 Number of observations in data file 10 Current missing variable code 1.000000000000000E+31 B34S Matrix Command. d/m/y 29/ 7/07. h:m:s 8:37:30. Ordinary Least Squares Estimation - Real*16 & QR Variable Lag Coefficient SE t X16 0 0.96522007E-01 2.0278033 0.47599294E-01 CONSTANT 0 0.99034780E+10 0.20278033E+10 4.8838454 Beta from VPA - default 64 digits accuracy BETA_VPA= Vector of 2 elements VPA - FM .965220M-1 .990348M+10 9.65220067890258342009546806150744940551143108123572854800000M-2 9.90347799865209574142761316341563647882994345591008434102590M+9 Beta from VPA - 200 digits accuracy BETA_VPA= Vector of 2 elements VPA - FM .965220M-1 .990348M+10 9.6522006789025834200954680615074494055114312994576814995573154883862781 971061687532225195211872707883443648855216365597479743619588764846877171 146998987325841273459533669904135897504647183082754700000M-2 9.9034779986520957414276131634156364788299434544608648119274223099391811 078965853733763483342081454150397522719740321509191330273731665955348598 768449388118628589173691417115419955283479769263019114930M+9 Display More digits to see what real*16 has 9.90347806584471095667600829609938E+09 9.65219395964106601701812011968070E-02 Compare with reading into real*8 then to real*16 9.90347799865209574142752881434979E+09 9.65220067890258342010390296808680E-02 Beta from VPA where character read into VPA This is the right way to attack the problem For gain compare with real*16 where character read 9.9034780658447109566760077568540548388203264036962318951564333468144967 976254007455322706347553966533211125389976555538803813972707424451639716 305267784566221628731949001317005969203917343089228659810M+9 9.6521939596410660170181740442134299509971673602685383945928568978486775 297722321536118749107570909723269206071150230416216962175365207909055592 648697631862131571369199609764191317652106807309346900000M-2 Beta from VPA where vpa converted from real*8 9.9034779986520957414276131634156364788299434544608648119274223099391811 078965853733763483342081454150397522719740321509191330273731665955348598 768449388118628589173691417115419955283479769263019114930M+9 9.6522006789025834200954680615074494055114312994576814995573154883862781 971061687532225195211872707883443648855216365597479743619588764846877171 146998987325841273459533669904135897504647183082754700000M-2 Difference of coef # 1 and # 2 Shows effect of not reading in correct precision 6.7192615215248394593438418359990382949235367083229011036875315689728815 372155922300547251238281360267023623402961248369897575849629111753681839 644759303955825758420158601392043757382620954488000000000M+1 -6.719261517403077294017294019454514263939189143104964458590537600667333 936599610644610430179816017444278406613518126278144422355693782157849830 1355463709702090334060139944579852540375773407800000000000M-8 This example shows that reading into real*8 first then converting to real*16 and or VPA causes a loss of accuracy in difficult problems. 25. Order of Operation issues. The command call bestreg (y x1 x2 x3 x4 :crit -3 :print); will not work as intended since the parser will attempt the operation crit-3. There are two ways to proceed. One it to pass a variable mthree=-3; call bestreg (y x1 x2 x3 x4 :crit mthree :print); A more elegant way is to force creation of a temp variable call bestreg (y x1 x2 x3 x4 :crit sfam(-3) :print); 26. Power of a matrix, array and a vector If x is an array the statements x=array(2:1 2 3); xnew=x**2.; produce an array of 1 4 9 If x is defined as a vector, x=vfam(x); xvnew =x**2.; produces 14 or the same as x*x If x is a 2-D array, then x=array(3,3:1 2 3 4 5 6 7 8 9) xnew=x*2 gives a matrix 1 4 9 16 25 36 49 64 81 If x is a matrix, then x_matnew = x**3.; is the same as x*x*x However the statement x_matnew=x**3.5; cannot be done this way. The solution relies on decomposing x as v*s*inv(v) where v are the eigenvectors of x and s is a diagonal matrix with the eigenvalues (e) of x along the diagonal. If x is a general matrix (not positive definate) these are possibly complex. The calculation for x_matnew is x_matnew = v * ss * inv(v) where ss is a diagonal matrix where ss(i,i)=e(i)**3.5. Some of these ideas are illustrated in the example file math_power in matrix.mac which is shown below. /; /; Matrix, array and Vector Power tests /; b34sexec matrix; aa=array(4:1 2 3 4); va=vfam(aa); aa16=r8tor16(aa); va16=r8tor16(va); caa=complex(aa); cva=complex(va); qcaa=qcomplex(aa16); qcva=qcomplex(va16); saa=sngl(aa); sva=vfam(saa); vpa_aa=vpa(aa); vpa_va=vpa(va); saa=sngl(aa); sva=vfam(saa); /; /; array and vector tests /; call print(saa, saa*saa, saa**sngl(2.)); call print(sva, sva*sva, sva**sngl(2.)); call print(aa16, aa16*aa16, aa16**r8tor16(2.)); call print(va16, va16*va16, va16**r8tor16(2.)); call print(caa, caa*caa, caa**complex(2.)); call print(cva, cva*cva, cva**complex(2.)); call print(qcaa, qcaa*qcaa, qcaa**qcomplex(r8tor16(2.)), qcva, qcva*qcva, qcva**qcomplex(r8tor16(2.))); call print(vpa_aa ,vpa_aa *vpa_aa, vpa_aa**vpa(2.), vpa_va ,vpa_va *vpa_va, vpa_va**vpa(2.)); call print('2 D Tests ++++++++++++++':); aa=array(3,3:1. 2. 30. 4. 5. 60. 7. 8. 90.); va=vfam(aa); aa16=r8tor16(aa); va16=r8tor16(va); caa=complex(aa); cva=complex(va); qcaa=qcomplex(aa16); qcva=qcomplex(va16); saa=sngl(aa); sva=vfam(saa); vpa_aa=vpa(aa); vpa_va=vpa(va); saa=sngl(aa); sva=vfam(saa); /; /; array and matrix /; call print(saa, saa*saa , saa**sngl(2.)); call print(sva, sva*sva , sva**sngl(2.)); call print( aa, aa* aa , aa**2.); call print( va, va* va , va**2.); /; /; This calculation will draw an error /; /; call print( va**2.5); /; call print(aa16, aa16*aa16, aa16**r8tor16(2.)); call print(va16, va16*va16, va16**r8tor16(2.)); call print(caa ,caa *caa, caa**complex(2.)); call print(cva ,cva *cva, cva**complex(2.)); call print(cva ,cva *cva*cva cva**complex(3.)); call print(qcaa, qcaa*qcaa, qcaa**qcomplex(r8tor16(2.))); call print(qcva, qcva*qcva, qcva**qcomplex(r8tor16(2.))); call print(vpa_aa ,vpa_aa *vpa_aa, vpa_aa**vpa(2.)); call print(vpa_aa ,vpa_aa *vpa_aa*vpa_aa, vpa_aa**vpa(3.)); call print(vpa_va ,vpa_va *vpa_va, vpa_va**vpa(2.)); call print(vpa_va ,vpa_va *vpa_va*vpa_va, vpa_va**vpa(3.)); /; /; this is an error since no eig capabilkity for vpa /; call print(vpa_va ,vpa_va *vpa_va*vpa_va, vpa_va**vpa(3.1)); /; /; Non integer power for complex*16 & complex*32 /; call print(eig(cva),eig(qcva)); call print('Non integer power complex*16', cva**complex(3.5)); call print('Non integer power complex*32', qcva**qcomplex(r8tor16(3.5))); /; /; Slow way to get an inverse /; * Test case for Real Matrix from IMSL Math (10) pp 295-297; * using eig( ) analysis to get inverse; x=complex(matrix(3,3:8.,-1.,-5.,-4., 4.,-2.,18.,-5.,-7.)); call print('X Matrix',x); call print(inv(x), x**complex(-1.)); call print(inv(c16toc32(x)), c16toc32(x)**c16toc32(complex(-1.))); b34srun; 26. Level issues The default command level is 100. If a subroutine is called the level is set to 101. If another routine is called, the level becomes 102. LEVEL=1 is the global level. Variables at the global level must be moved to the local level to be used because of assingment level issues. For example xlocal=xglobal; makes a local copy of xglobal. As an alternative, the command call makeglobal(x) Puts X at level 1. X can be passed to a subroutine where a local copy is automatically made. Any changes are placed back at the global level. As another alternative, the command call makelocal(x); will make x local. As notes, if X is global, statements such as y=x*x; will place x*x at the local level. /; /; Global variable moved to a subroutine and made local. /; Calculations made and the result put back at global level. /; b34sexec matrix; y=array(:10. 20.); call makeglobal(y); call print(level(v)); call print(level(y)); subroutine useit(x); call names(all); call print(x); x=x+100.; return; end; call useit(y); call names(all); call print(y); b34srun; Matrix Programming Language key words CALL Call a subroutine call print('This is a string'); calls the print command. B34S MATRIX commands that do not return an argument across an equals are executed by the CALL sentence. The CALL sentence first looks in named storage for a routine with this name. If this is not found, then the built-in routines are used. While it is possible to have a user routine with the same name as a built in routine, this is not a good idea and can caluse unexpected errors. CONTINUE go to statement name continue; is a go to statement target. Note name must be le 8 characters Example: if(x.eq.0.0)go to test; call print('x is greater than 0.0'); test continue; DO Starts a do loop do i=1,10; Begins a DO loop. An alternative is: do i=1,10,1; Another alternative is for i=1,10; Notes on do loops: The code: do i=1,10; do j=i,10; s(i,j)=b(i)*a(j); enddo; enddo; is valid. As written, both do loops will completely execute. If in place of the above, the code had been: do i=1,10; do j=i,10; s(i,j)=b(i)*a(j); if(s(i,j).le..0001)go to done; enddo; done continue; call print('This is the end of loop one'); enddo; the results would not be as intended since B34S does not know to return to the statement do i=1,10; and will instead try to continue with the statement do j=i,10; since it may not "know" the exact location of the inner loop enddo; statement because it may not have found it. The solution is to manually terminate the inner loop by setting j outside the range. Corrected code would be: do i=1,10; do j=i,10; s(i,j)=b(i)*a(j); if(s(i,j).le..0001)go to done; enddo; done continue; j=0; call print('This is the end of loop one'); enddo; Example: do i=1,n; x(i)=y(i+1); enddo; A faster code would be i=integers(1,n); j=i+1; x(i)=y(j); Notes on DO loop variables. In Fortran it is not recommended that DO loop variables be used outside the loop. In B34S b34sexec matrix; do i=1,2; call print('in loop i = ',i); enddo; call print('out of loop i =,i); b34srun; Produces i=3 at the "out of loop" position. This is the same as Fortran. To check compile c test fortran do i=1,2 write(6,*)'In loop i was ',i enddo write(6,*)'Out of loop i was ',i end Note how closely B34S follows the Fortran standard and the language. DOWHILE Starts a dowhile loop dowhile(x.gt.0.0); Starts a DOWHILE loop. Example: b34sexec matrix; sum=0.0; add=1.; count=1.; tol=.1e-6; dowhile (add.gt.tol); oldsum=sum; sum=oldsum+((1./count)**3.); count=count+1.; add=sum-oldsum; enddowhile; call print('Sum was ',sum:); call print('Count was ',count); b34srun; Warning: Be sure you do not have an infinate loop. Note: The dowhile loop will cycle at least one time since the test is made at the end of the loop when enddowhile is hit. If this is not desired, place the dowhile loop inside an if statement. The below listed example checks if there is full rank for a bootstrap of a ls2 model. if(rank_zpx.lt.nocols(%zpx).or.rank_zpz.lt.nocols(%zpz)) then; dowhile (itest.lt.100.and.(rank_zpx.lt.nocols(%zpx) .or.rank_zpz.lt.nocols(%zpz))); j=booti(norows(%x)); newy=%y(j); newx=%x(j,); newz=%z(j,); %zpx=transpose(newz)*newx; %zpz=transpose(newz)*newz; rank_zpx=rank(%zpx); rank_zpz=rank(%zpz); itest=itest+1; call print('bottom of dowhile':); enddowhile; endif; ENDDO ENDS a DO loop enddo; Ends a DO loop. next i; Can be used in place of enddo; Example: do i=1,n; x(i)=y(i+1); enddo; ENDDOWHILE ENDS a dowhile loop enddowhile; Ends a DOWHILE loop. Example: b34sexec matrix; sum=0.0; add=1.; ccount=1.; count=1.; tol=.1e-6; /$ outer dowhile does things 2 times call outstring(2,2,'We sum until we can add nothing!!'); call outstring(2,4,'Tol set as '); call outdouble(20,4,tol); call echooff; dowhile(ccount.ge.1..and.ccount.le.3.); sum=0.0; add=1.; count=1.; dowhile(add.gt.tol); oldsum=sum; sum=oldsum+((1./count)**3.); count=count+1.; call outdouble(2,6,add); add=sum-oldsum; enddowhile; ccount=ccount+1.; call print('Sum was ',sum:); call print('Count was ',count); call compress(1000); enddowhile; b34srun; END End of a program, function or Subroutine. end; Ends a PROGRAM, SUBROUTINE or FUNCTION. Example: subroutine test(x); call print('The mean of x is',mean(x)); return; end; EXITDO Exit a DO loop exitdo; Exits a do loop. Example: b34sexec matrix; call echooff; do j=1,4; do i=1,10; if(i.eq.8)exitdo; if(i.ge.0)then; call print('at 1 in if i was ',i:); if(i.ge.4)exitif; call print('at 2 in if Should never be gt 3 i was ',i:); endif; call print('in do loop ',i); enddo; enddo; b34srun; EXITIF Exit an IF loop exitdo; Exits a do loop. Example: b34sexec matrix; call echooff; do j=1,4; do i=1,10; if(i.eq.8)exitdo; if(i.ge.0)then; call print('at 1 in if i was ',i:); if(i.ge.4)exitif; call print('at 2 in if Should never be gt 3 i was ',i:); endif; call print('in do loop ',i); enddo; enddo; b34srun; FORMULA Defive a recursive formula. formula x=y(t-1); Defines a formula. Formulas are only executed for observation t in a SOLVE statement. Formula definations are saved at level 2. Formula results are saved at the current level and can be used in the usual analytical statements provided that they have been executed by a solve statement. More extensive help in given later in the help document. Brief example: * archvar and resid start out as variables =0.0; * formula statement updates ONLY for obs t; * For t=2, this value of u is used to get archvar; archvar=array(norows(y):); resid =array(norows(y):); formula archvar = a0 + a1 * (resid(t-1)**2.) ; formula resid=y(t) - b1 - b2*x1(t) - b3*dsqrt(archvar(t)); solve( archlogl=(-.5)*(dlog(archvar(t))+((resid(t)**2.)/archvar(t)) :range 2 norows(y) :block archvar resid); FOR Start a DO loop for i=1,10; Alternate do loop setup. Example: for i=1,n; x(i)=y(i+1); next i; is the same as do i=1,n; x(i)=y(i+1); enddo; NEXT i End of do loop next i; Alternate end of DO loop Example: for i=1,n; x(i)=y(i+1); next i; GO TO Transfer statement go to n; Transfers control to statement n CONTINUE; Note n must be le 8 characters Example: if(x.eq.0.0)go to test; call print('x is greater than 0.0'); test continue; Note: Statements such as if(k.eq.0)then; * statements here ; if(jj.gt.0)go to done; endif; should be avoided since the statement endif; will not be found and the # of if statements will be exceeded. Better code is bad=0; if(k.eq.0)then; * statements here; bad=1; endif; if(bad.eq.1)go to done; or if(k.eq.0)then; * statements here; if(jj.gt.0)exitif; endif; FUNCTION Beginning of a function. function somename(args); is the first line of a user function. Functions can have functions as arguments and themselves can be used as arguments. The command call compress; will be ignored if found in a function or a subroutine or a program called by a running function. Note: Repeated calls to user functions use up parse space. Thus for heavy computing, user subroutines should be employed in place of user functions. If functions are used outside subroutines, the option adjust_parse can be used. This "undocumented" command will NOT work if a function is used as an argument inside a subroutine call. Hence the use of the option adjust_parse should not be generally employed. As a second place alternative, the parse tree can be expanded by use of b34sexec options maxnsent(600) maxntoken(5000); b34srun; Examples: function tt(y); t=sum(y); return(t); end; testmean=tt(y)/dfloat(norows(y)); call mysub(tt(y),x,z); IF( ) Beginning of an IF structure Note: The WHERE statement operates on non scalar objects while the IF statement operates on scalar objects. if(x.eq.9)y=dabs(x); Simple IF statement. X must be a scalar. If the mask JJ (must be 0.0 or 1.0) was set the code: jj(1)=0.0; jj(2)=1.0; if(jj(i))call print('i was 2'); can be used. The commands NLPMIN1, NLPMIN2 and NLPMIN3 use mask technology. The statements: if(x.eq.9)then; call stop; endif; will STOP the program and get out of the MATRIX command if x=9. Note that IF statements ( ) must resolve to 0.0 or 1.0. A check for 0.0 of the form if(x.ne.0.0)y=p/x; will work in versions of B34S since November 7, 2004. An alternative is: if(x.ne.0.0)then; y=p/x; endif; MASKS are a feature related to IF statements but much faster. Since a logical expression resolves to be 0.0 or 1.0, a mask can be built with a expression such as mask = x .gt. 1.0 ; Masks are a way to vectorize what would be an IF. For tyeh below listed example note that x is an array, not a vector. For example: b34sexec matrix; call print('If X GE 0.0 y=20*x. Else y=2*x':); x=rn(array(20:)); y= (x.lt.0.0)*(x*2.0) + (x.ge.0.0)*(x*20.); call tabulate(x,y); b34srun; will run faster than b34sexec matrix; call print('If X GE 0.0 y=20*x. Else y=2*x':); x=rn(array(20:)); do i=1,norows(x); if(x(i).lt.0.0)y(i)=x(i)*2.0; if(x(i).ge.0.0)y(i)=x(i)*20.; enddo; call tabulate(x,y); b34srun; due to DO loop overhead. Note: Statements such as if(k.eq.0)then; * statements here ; go to done; endif; should be avoided since the statement endif; will not be found and the # of if statements will be exceeded. Better code is bad=0; if(k.eq.0)then; * statements here; bad=1; endif; if(bad.eq.1)go to done; Examples of code to change an element of an array using a mask. x=integers(1,10); xx=x; /$ /$ Note that Where sets one element to 99 rest to 0.0; /$ where(x.eq.5)x=99; /$ This may not be what is desired. /$ This is the right way to do calculation using masks xx=(xx.eq.5)*99+(xx.ne.5)*xx; call print(x,xx); ENDIF End of an IF( )THEN structure endif; Must be the end of an if( )then; structure. Example: if(x.eq.0.0)then; y=10.; v=y*dsin(q); endif; PROGRAM Beginning of a program, program somename; begins a program. Programs use global variables. Example: program doit; call loaddata; call olsq(x,y :print); call graph(%res); return; end; RETURN( ) Returns the result of a function. return(result); Next to last statement in a user function. Example: function tt(y); t=sum(y); return(t); end; RETURN Next to last statement before end. return; must be the next to last statement in a PROGRAM, SUBROUTINE or FUNCTION. For a function the form return(value); is used. Example: program doit; call loaddata; call olsq(x,y :print); call graph(%res); return; end; SOLVE Solve a recursive system of equations. solve( ); solves a recursive system. solve(vv=x(t) :range 1 10); Solves recursively an expression. If formulas are involved, the form is solve(vv=x(t) :range 1 10 :block x); Note: :range i j is OK :range i, j is OK :range i, norows(gasout) is ok :range (i+6), j is not ok For more detail see extensive example below: b34sexec matrix; /$ Unlike RATS, SOLVE and FORMULA statements can /$ refer to themselves recursively n=1000; v=1.0; ar1=array(n:)+missing(); ar1(1)=99.+rn(v); solve(ar1=ar1(t-1)+rn(v):range 2 n); call graph(ar1); call tabulate(ar1); b34srun; SUBROUTINE Beginning of subroutine. subroutine somename(args); is the start of a subroutine. Subroutines use local variables. Example: subroutine test(x); call print('The mean of x is',mean(x)); return; end; WHERE( ) Starts a where structure. Note: The WHERE statement operates on non scalar objects while the IF statement operates on scalar objects. where(x.gt.0.0)y=x; Assuming x exists. where elements of x > 0.0 y=x, otherwise y=oldvalue. Variables x and y must be the same structure. The "mask" is done at the copy step. Commands of the form where(s.gt.0.0)y=dsqrt(s); will not work as intended since the dsqrt(s) is done BEFORE the mask is applied. Only simple replacement is allowed. Statements such as where(x.ge.0.0)z(,1)=q; are not allowed since z(,1) is a temp variable. The statements where(x.ge.0.0)x=missing(); where(x.ge.10.)x=dqsrt(20.); are allowed but care must be used. Since x and missing() and dsqrt(20.) are not the same structure, here if ( ) is "false" x is set = 0.0. For example, in the first statement where x ge 0.0, x is set to missing(). If x lt 0, x is set to 0.0. The second statement sets x to dsqrt(20.) if x was ge 10. Otherwise x is set to 0.0. Note that depending on whether x existed or not the statement does not return the same vector. This is an explicit design decision. Like Speakeasy, where( ) returns the existing value if the logical statement is false and zero if the variable did not exist! Assume x does not exist. Given where(x.ge.0.0)newx=missing(); Here if x ge 0.0, then newx = missing(), otherwise newx = 0.0. Logical masks are an alternative to some of the where capability. For example: x=a.gt.y; Here where a is > y, x = 1.0, x = 0.0 otherwise. Example: /; Here for the first where( ) the two objects /; across the equals sign are not the same structure /; If the ( ) is false x2bad resolves to 0.0 whether or /; not it existed prior to the where( ) being found. /; /; The second where( ) has objects the same structure across /; the =. Both objects exist. Here the old x value is /; maintained. The logic here is /; test = x*(x.ne.y)+dummy*(x.eq.y) b34sexec matrix; x=array(:1,-2,3,-4,5,-6,7,-8,9,-10); y=array(:0,-2,1,-4,6,-6,2,-8,5,-10); x2bad=x; x2good=x; dummy=array(norows(x):)+ -9999.; where(x.eq.y)x2bad =-9999.; where(x.eq.y)x2good =dummy; test = (x*(x.ne.y))+ (dummy*(x.eq.y)); call tabulate(x,y,dummy,x2bad,x2good,test); b34srun; Edited Results are: => X=ARRAY(:1,-2,3,-4,5,-6,7,-8,9,-10)$ => Y=ARRAY(:0,-2,1,-4,6,-6,2,-8,5,-10)$ => X2BAD=X$ => X2GOOD=X$ => DUMMY=ARRAY(NOROWS(X):)+ -9999.$ => WHERE(X.EQ.Y)X2BAD =-9999.$ => WHERE(X.EQ.Y)X2GOOD =DUMMY$ => TEST = (X*(X.NE.Y))+ (DUMMY*(X.EQ.Y))$ => CALL TABULATE(X,Y,DUMMY,X2BAD,X2GOOD,TEST)$ Obs X Y DUMMY X2BAD X2GOOD TEST 1 1.000 0.000 -9999. 0.000 1.000 1.000 2 -2.000 -2.000 -9999. -9999. -9999. -9999. 3 3.000 1.000 -9999. 0.000 3.000 3.000 4 -4.000 -4.000 -9999. -9999. -9999. -9999. 5 5.000 6.000 -9999. 0.000 5.000 5.000 6 -6.000 -6.000 -9999. -9999. -9999. -9999. 7 7.000 2.000 -9999. 0.000 7.000 7.000 8 -8.000 -8.000 -9999. -9999. -9999. -9999. 9 9.000 5.000 -9999. 0.000 9.000 9.000 10 -10.00 -10.00 -9999. -9999. -9999. -9999. *********************************************** Documentation of built-in commands called by CALL command. For futher examples, see problems in matrix.mac ACEFIT Alternating Conditional Expectation Model Estimation call acefit(y x[orderable] z[orderable]{2} :options); Implements the ace algorithm under matrix to provide estimation of ACE (alternating condition expectation) models following work by Breiman, L and Friedman J. (1985). Chapter 7 of Hastie and Tibshtiani (1990) provides a good reference. For another approach see the GAMFIT command. The ACEFIT command optionally can use the AVAS (Additivity and Variance Stabilization) approach. As noted in Hastie-Tibshirani (1990 page 193) "The AVAS procedure seeks transformations that achieve additivity and a homogenious variance and is more directed towards regresison problems than ACE." While AVAS may be more desirable for regresison models, its theoretical support is not as strong as for the ACE approach. - Faraway, Julian. "Extending the Linear Model with R" 2006, New York: Chapman & Hall/CRC - Hastie, T., R. Tibshirani and J. Friedman "The Elements of Statistical Learning: Data Mining, Inference, and Prediction." 2001 New York: Springer. - Breiman, L. and Friedman J. "Estimating Optimal Transformations for Multiple Regression and Correlation (with discussion),"Journal of American Statistical Association, 80, (1985) 580-619. - Hastie, T. J. and Tibshirani, R. J. (1990) "Generalized Additive Models," New York: Chapman and Hall Recent papers in the area include: - Fan, Jianqing and Jiancheng Jiang. "Nonparametric Inference for Additive Models," Journal of the American Statistical Association, 100, # 471 (2005) 890-907. where the generalized likelighood ratio test (GLR) is discussed. This test has not been implemented but can be implemented by interested users. Assume y = f(x1,...,x2) gamfit transforms the independent variables while acefit transforms both independent and dependent variables. Hastie and Tibshirani make the point that a model of the form y=exp(x1+x2**2)e cannot be estimated in additive form by gamfit but a simple additive model can be found that describes log(y). For example log(y)=x1+x2**2 + ln(e) Acefit procedure: Assume G(y) is the transformed dependent variable. The ACE procedure forces var{G(y)} = 1 and minimizes E{G(y)-f(X)}**2 subject to var{G(y)}=1. Assume one input variable x. yhat = ((G(y))**-1){a+sum(f(x))} where f(x]=) is the transformation for the input variable x and the transformation for the dependent variable is assumed to be invertable.. Logic of ACE: 1. For fixed G, the minimizing f is f(x)=E{G(y)|x} 2. For the minimizing f in step 1, the minimizing G is G(y)=E(f(x)|Y}. In this step new_(y)=estimated_g(y)/var{estimated_g(y)}**.5 Steps 1 and 2 alternate after assuming G(y) so that it has unit variance at each step to avoid a trivial zero solution. In words, the ace algorithm alternates between smoothing g(y) on x to get a new f(x), and f(x) on y to get a new g(y), until the mean-squared error does not change. Since the B34S implementation of ace saves all yhat vectors and residual vectors, the user can select the best model to use. The program call ace_ols; will save %best as a integer for the best model which allows the user to extract the transformed y and x matrix data and the %yhat vector for this model. Graphing of the transformed data can provide information concerning nonlinearity in the model. The below listed examples show one criteria using the sum of squares of the residuals. However the user may want to weight the later performance more than the earlier performance of the model. The code to select best ace model (shown below) is placed after the call acefit command: /; Get best model call ace_ols; ace_res=%res(,%best); If the OLS model is not desired the call call ace_ols2(%k,%ns,%ssres,%ty,%tx,%best,%ybest,%xbest, iprint); can be used. To test the y variable transformation check to see if is_1= variancevariance(%besty)* (dfloat(norows(%besty)-1)/dfloat(norows(%besty))); is 1.0. Logic of AVAS In theory AVAS is more suitable for regession models in that it involves an asymptotic variance stabilizing transformation. For more detail see Stokes (200x) and Faraway (2006, 244-246). %tx and %ty can be studied and compared between the two procedures. Variables created %res - Residuals saved for last %nob %ns matrix %y - Y variable %yhat - Predicted y saved as a %nob,%ns array matrix %yvar - Y variable name %names - Names in Model %lag - Lag %vartype - Variable type %dist - Error Distribution %nob - Effective number of observations. %k - Number of right hand side variables. %ns - Number of passes. %rsq - Vector of R**2 for ns runs. This is the r**2 for the %ty series. The R**2 for y is given in the printout, if requested, but can be calculated from 1-(%ssres / variance(y)*dfloat(norows(y)-1)) %ssres - Vector of residual sum of squares. %best=imin(%ssres); %ty(n,ns) - Transformed y. %tx(n,p,ns) - Transformed x. [ ] Specifications. Allowed values are: order e => usual case. This is default for y and x. circular => periodic in range 0.0 to 1.0 with period 1. monotone => Transform must be monotone linear => transform is linear cat => not orderable. If the transformation is supplied, then the corresponding codes are fix_order fix_circular fix_monotone fix_linear fix_cat Examples: Call acefit(y x1 x2[order] x3[order]{1} x4[order]{1 to 6} :print); Note: If lags are present then based on minimum lag the following is saved. This will not occur if future right hand side variables are present and :holdout is not present. %xfobs - Observation number. %xfuture - Same as %x but for out of sample data that is available. Note: while x1 is allowed x1{1} is not since [ ] is missing. Options supported :print Show output :itprint Show Iterations :avas Use AVAS approach to get transformations. This model does not forecast at this time. It is more a diagnostic tool. :span r1 Sets span. Default = 0.0. Set as a fraction of observations in window. 0.0 => automatic (variable) span selection. For small samples (n < 40) or if there are substantial serial correlations between obserations close in x - value, then a prespecified fixed span smoother (span > 0) should be used. Reasonable span values are 0.3 to 0.5. :alpha r2 Sets alpha. Default = 0.0. Controls high frequency (small span) penality used with automatic span selection (base tone control). alpha.le.0.0 or alpha.gt.10.0 => no effect.) :ns ns Number of solutions. Default = 3. :ty ty Supplied y transformation. ty(n,ns) :tx tx(n,p,ns) Supplied x transform :maxit i1 Maximum number of iterations. Default=20 :nterm i2 Maximum number of terminal iterations. Default = 3 :tol delrsq Sets termination threshold. Default=.1e-3. Iteration stops when R**2 changes less than delrsq in i2 iterations. :holdout n Sets number of observations to hold out :sample mask Specifies a mask real*8 variable that if = 0.0 drops that observation. Note: :sample cannot be used with :holdout. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better choice is the :holdout option. :weight w Set weight for each obs. Series w must have same # of observations as left hand variable. :spans r3 Sets a three element array. Spans values are for the three running linear smoothers. spans(1) : tweeter span. spans(2) : midrange span. spans(3) : woofer span. Default => array(:.05,.2,.5) This parameter should not be adjusted under normal circumstances. :savex Saves the X matrix in %X. call acefit(y x[order]{1 to 6} z[cat] :print); The model specification involves specification of the type of variable and optionally a lag or lags. The model specification allows the lags to be set in the command. Only vectors can be supplied in this release. If no[ ] is supplied, [order] is assumed. The specification call acefit(y y[order]{1} x[order]{0 to 3} z[order]{1} )$ is the same as call acefit(y y[order]{1} x[order] x[order]{1} x[order]{2} x[order]{3} z[order]{1})$ :getmodel 'filename' Saves model discription variables If getmodel is found, no estimation is performed. :savemodel 'filename' The default name is 'acemod.psv' :forecast xmatrix => Allows users to supply observations of the right hand side variables outside the sample period so that forecasts can be calculated. The same number of observations must be supplied for all right hand series. Due to the way that splines are calculated, it is imperative that any values of the x variables NOT lie outside the ranges of the original data. The user must have save the model workspace if :forecast is the only option. The forecast sentence produces the %fore variable and the %foreobs variable. As of this time :forecast works only for ACE models. :fmodel integer Sets model number to forecast. Default=1 Variables saved if :savemodel is set: %X, %K, %YVAR, %NAMES, %LAG, %NS, %TY, %TX, %Y, %NOB, %RSQ, %RES, %YHAT, %LF, %M, %Z, %F, %T Many of these values have use only in forecasting. Important supplementary routines program ace_ols; /; /; Extracts "best" ACE Model - saves transformed y in /; %ybest and saves transformed x in %xbest %best = index /; /; Obtains %k, %ssres %ty and %tx from last acefit model /; /; Experimental version 19 February 2006 /; /; For reference and motivation see Faraway (2006) P242 /; Can be called as a program or as a subroutine using iprint=1; call aceols2(%k,%ns,%ssres,%yhat,%ty,%tx,%best,%ybest, %xbest, iprint,%ace,%y); call print(' ':); call print('Test sum of squares ',sumsq(%y-%ybest):); return; end; subroutine aceols2(k,ns,ssres,yhat,ty,tx,best,ybest,xbest, iprint,ace,y); /; /; Extracts best ACE model data and optionally shows /; implied OLS models /; /; Let y* = theta(y) /; /; 1. y* = f( smoothed x) /; 2. y* = f( smoothed x without constant) /; 3. y = f( smoothed x ) /; /; Y can be forecasted two ways: /; -using inverse of y filter /; -using equation 3. /; /; Can be called by ACE_OLS or directly /; /; k => # right hand side variables /; ns => # of ACE models attempted /; ssres => array with ns elements showing residual sum /; of squares /; of ACE model /; yhat => nob,ns 2d object containing y hat /; ty => transformed y /; tx => nob,k,ns of nob * (k*ns) object containing /; transformed x /; best => # of ACE model that has minimum e'e /; ybest => nob element transformed y of "best" model /; xbest => nob,k matrix of transfrmed x elements of /; "best" model /; iprint => =0 only calculate, NE 0 => print OLS models /; of ybest = f(xbest) /; ace => Set by call acefit. use %ACE /; /; Experimental version 19 February 2006 /; /; For reference and motivation see Faraway (2006) P242 /; program ace_plot; /; /; Plot ACE Transformations for BEST Model /; subroutine aceplot2(y,ybest,xbest,xorig,names,lag,ace); /; /; Show Ace Transforms in Graphs /; See Faraway (2006) page 243 /; /; y => Usually %y /; ybest => Best ytransform /; xbest => best xtransform /; xorig => %x from :savex /; names => %names /; lag => %lag /; ace => %ACE Examples of using supplementary routines. Shows GAM vs ACE model. /; /; ACE Plots /; b34sexec options ginclude('b34sdata.mac') member(b_ozone); b34srun; b34sexec matrix; call loaddata; call echooff; call load(ace_ols ); call load(ace_plot); iholdout=10; call gamfit(ozone vh[predictor,3] wind[predictor,3] humidity[predictor,3] temp[predictor,3] ibh[predictor,3] dpg[predictor,3] ibt[predictor,3] vis[predictor,3] :holdout iholdout :print); call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] :holdout iholdout :ns 4 :savex :print); call ace_ols; call ace_plot; b34srun; Examples: b34sexec options ginclude('b34sdata.mac') member(gam); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call echooff; call acefit(y[cat] age[order] start_v[order ] numvert[order] :print); call gamfit(y age[predictor,3] start_v[predictor,3] numvert[predictor,3] :link logit :dist gauss :maxit index(2000,1500) :tol array(:.1d-13,.1d-13)); b34srun; /; /; ACEFIT Problem showing plots for various NS values /; b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call olsq(cpeptide age bdeficit : print); call echooff; call acefit( cpeptide[order ] age[order] bdeficit[order] :maxit 20 :nterm 10 :ns 2 :tol .1e-8 :print); call names(all); call tabulate(%rsq,%ssres); call print(%y); call print(%yhat,%res); do i=1,%ns; call graph(%res(,i)); enddo; b34srun; /; /; Experimental AVAS Option /; b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call olsq(cpeptide age bdeficit : print); call echooff; call acefit( cpeptide[order ] age[order] bdeficit[order] :avas :maxit 20 :nterm 10 :tol .1e-8 :print); call names(all); call tabulate(%rsq,%ssres); call print(%y); call print(%yhat,%res); do i=1,%ns; call graph(%res(,i)); enddo; b34srun; /; /; ACEFIT Vs GAMFIT on Gas Data /; b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(ace_ols); maxlag=6; call olsq(gasout gasout{1 to maxlag} gasin{1 to maxlag} :print); ols_res=%res; call acefit(gasout gasout[order]{1 to maxlag} gasin[order]{1 to maxlag} :print); call ace_ols; ace_res=%res(,%best); call gamfit(gasout gasout[predictor,3]{1 to maxlag} gasin[predictor,4]{1 to maxlag} :print); gam_res=%res; call marspline(gasout gasin{1 to 4} gasout{1 to maxlag} :nk 40 :mi 2 :print); mars_res=%res; call graph(ols_res,gam_res,ace_res,mars_res :nolabel); Models=c8array(:'OLS','ACE','GAM','MARS'); fit =array(:sumsq(ols_res),sumsq(ace_res), sumsq(gam_res),sumsq(mars_res)); call tabulate(models,fit); b34srun; Forecasting ACE vs MARS /; /; ACE validation of Forecasting - /; Feeds data back in to prove forecast /; Out of sample Forecast done /; /; Data set used was studied in Faraway (2006) page 241 b34sexec options ginclude('b34sdata.mac') member(b_ozone); b34srun; b34sexec matrix; call loaddata; call echooff; iholdout=80; idoace =1; iacefore=1; idomars=1; /; ACE +++++++++++++++++++++++++++++++++++++++++++++++++++ if(idoace.ne.0)then; call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] doy[order ] :savemodel :holdout iholdout :ns 1 :savex :print); %x_ace=%x; %yhatt=%yhat; /; First forecast the complete sample Check sum of squares if(iacefore.ne.0)then; call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] doy[order ] :holdout iholdout :getmodel :forecast %x_ace :print); call print('%fore and %yhatt have to be the same':); call tabulate(%foreobs %fore,%yhatt,ozone); jj=integers(norows(ozone)-iholdout); %actual=ozone(jj); call print(' '); call print('Test Error Sum of squares both ways ':); call print('Error sum of squares using yhat ', sumsq(%actual-afam(%yhatt)):); call print('Error sum of squares using %fore ', sumsq(%actual-afam(%fore )):); call print(' ':); /; Future forecast reloading the data if(iholdout.ne.0)then; call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] doy[order ] :holdout iholdout :getmodel :forecast %xfuture :print); jj=integers(norows(ozone)-iholdout+1,norows(ozone)); %actual=ozone(jj); %forace=%fore; call tabulate(%foreobs %fore,%actual); call graph(%forace,%actual :nolabel); endif; endif; endif; if(idomars.ne.0)then; call marspline(ozone vh wind humidity temp ibh dpg ibt vis doy :holdout iholdout :df 2. :nk 50 :mi 2 :savex :savemodel :print); call marspline(ozone vh wind humidity temp ibh dpg ibt vis doy :holdout iholdout :df 2. :nk 50 :mi 2 :savex :getmodel :forecast %xfuture :print); %formars=%fore; call tabulate(%foreobs %forace,%formars,%actual); call graph(%forace,%formars,%actual :nolabel); call print(' ':); ace_ess =sumsq(%actual-%forace ); mars_ess=sumsq(%actual-%formars); call print('ACE Out of Sample Sum of squares',ace_ess:); call print('MARS Out of Sample Sum of squares',mars_ess:); endif; b34srun; ACE_OLS Select best ACE Model Selects BEST ACE model and does a plot b34sexec options ginclude('b34sdata.mac') member(b_ozone); b34srun; b34sexec matrix; call loaddata; call echooff; call load(ace_ols ); call load(ace_plot); iholdout=10; call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] :holdout iholdout :ns 1 :savex :print); call ace_ols; call ace_plot; b34srun; ACE_PLOT ACE Plots Selects BEST ACE model and does a plot b34sexec options ginclude('b34sdata.mac') member(b_ozone); b34srun; b34sexec matrix; call loaddata; call echooff; call load(ace_ols ); call load(ace_plot); iholdout=10; call acefit(ozone vh[order ] wind[order ] humidity[order ] temp[order ] ibh[order ] dpg[order ] ibt[order ] vis[order ] :holdout iholdout :ns 1 :savex :print); call ace_ols; call ace_plot; b34srun; ACF_PLOT Simple ACF Plot subroutine acf_plot(series,nacf,title); /$ Simple ACF Plot routine /$ Series = Input series /$ nacf = # NACF and PACF /$ Title = Title /$ /$ DATA_ACF and DATA2ACF are a more complex commands /$ Use BJ_IDEN for ACF and PACF Listing /$ /$ *************************************** Example: b34sexec options ginclude('gas.b34')$ b34srun$ b34sexec matrix; call loaddata; call load(acf_plot); call acf_plot(gasout,24,'gasout'); b34srun; ADDCOL Add a column to a 2d array or matrix. call addcol(x,jbegin); Adds col at jbegin. The command for adding more than one col is: call addcol(x,jbegin,number); To add a col at right, give: call addcol(x); Note that jbegin and number are integer*4. ADDROW Add a row to a 2d array or matrix. call addrow(x,ibegin); Adds row at ibegin. The command for adding more than one row is: call addrow(x,ibegin,number); To add at bottom, give command: call addrow(x); Note that ibegin and number are integer*4. AGGDATA Aggregate Data under control of an ID Vector. call aggdata(id,x,newx,newid); Aggregates data in accordance with ID variable id Id variable to determine subgroup. ID must have been sorted and be lined up with x. x Series to be aggregated newx Mean of elements in group newid Group id of new series Series created %nelm # of observations in the group %nnzero # of non zero observations in the group %varx variance of elements in group Notes: The series %nelm allows one to subset the x easily. Example: b34sexec matrix; id=10.; x=20.1; call aggdata(id,x,newx,newid); call print(id,x,newx,newid,%nelm,%nnzero,%varx); id=array(6:10. 10. 11. 11. 11. 12.); x= array(6:1 2 3 4 5 6); call tabulate(id,x); call aggdata(id,x,newx,newid); call tabulate(newx,newid,%nelm,%nnzero,%varx); b34srun; ALIGN Align Series with Missing Data The align command trims series that are the same length initially but contain missing data. This command works for series like the goodrow( ) function works for a matrix. Align works for real*8, real*4 integer*4 and character*8 call align(x1, x2); After this command runs x1 and x2 are still the same length but now contain only non missing data. x1 and x2 must be 1D or 2d objects. To line up time series data starting in different periods use the command: call tslineup(ts1,ts2,ts3); which will place missing data where there are no observations. Example: b34sexec matrix; n=10; x=rn(array(n:)); y=rn(x); call tabulate(x,y); i=integers(1,n,2); x(i)=missing(); call tabulate(x,y); call align(x,y); call tabulate(x,y); b34srun; Advanced Time Series Example /; /; Shows line up and purging time series data. /; Due to possible missing data inside the series the /; timestart and timebase have not been set. However a /; date variable can be added to preserve the date of each /; observation /; b34sexec matrix; call tsd(:get c :file 'c:\b34slm\tsd3.tsd' :print :nomessage); call tsd(:get c96c :file 'c:\b34slm\tsd3.tsd' :print :nomessage); call tsd(:get cd :file 'c:\b34slm\tsd3.tsd' :print :nomessage); call names(:); /; do i=1,norows(%names%); /; call print(argument(%names%(i))); /; enddo; call names; call tabulate(c c96c cd); call tslineup(c c96c cd); call tabulate(c c96c cd); call align(c c96c cd); call tabulate(c c96c cd); call names; /; Using a date variable call clearall; call tsd(:get c :file 'c:\b34slm\tsd3.tsd' :print :nomessage :datename a1); call tsd(:get c96c :file 'c:\b34slm\tsd3.tsd' :print :nomessage :datename a2); call tsd(:get cd :file 'c:\b34slm\tsd3.tsd' :print :nomessage :datename a3); call names(:); /; do i=1,norows(%names%); /; call print(argument(%names%(i))); /; enddo; call names; call tabulate(c a1 c96c a2 cd a3); call tslineup(c a1 c96c a2 cd a3); call tabulate(c a1 c96c a2 cd a3); call align( c a1 c96c a2 cd a3); call tabulate(c a1 c96c a2 cd a3); call names; b34srun; ARMA ARMA estimation using ML and MOM. The ARMA command estimates univariate BJ models using ML and method of moments. Only one AR and MA factor is allowed. This command can be used to select relatively simple models from inside a user selected framework. If many series are to be filtered quickly, this command should be considered. The more complex command AUTOBJ will identify models with AR, MA, SAR and SMA factors. This command is based on the BJIDEN and BJEST routines available as B34S commands. The underlying code for this comamnd is the Peck Box Jenkins program that was developed under the supervision of George Box at UW in the late 60's and early 70's. Many accuracy improvements have been made by Houston H. Stokes. call arma(x :options): Estimates an ARIMA model on series x using the method of moments & nonlinear least squares. Forecasts and residuals can be calculated. Both unrestricted and restricted models can be fit. Box-Jenkins-Reinsel (1994) page 220 - 223 discusses the method of moments approach using the Newton-Raphson algorithm. The ARMA command uses the IMSL Library routines DN2LSE and DN2PE. Note: b34slf95.exe uses the IMSL library version 4 while b34sia32.exe uses the IMSL library version 6. It appears that the version 4 IMSL routines are more robust for reasons that are unknown. Options supported to estimate unrestricted models: :nar n - Sets n as the max AR order provided all terms up to n are to be estimated. In this case the keyword :arorder is not needed. :arorder ivec - Sets AR terms to be estimated for restricted model. :nar is not set in this case. :arparms rvec - Sets initial AR parameter values. Usually not required. :nma m - Set for max MA order provided all terms up to m are to be estimated. In this case :maorder is not needed. :maorder ivec - Sets MA terms to be estimated for restricted models. :nma is not set in this case. :maparms rvec - Sets initial MA parameter values. Usually not required. :relerr r - Stopping criterion for method of moments (MM) estimation. If r =0.0, default = 100*amach(4). :maxit n - Maximum number of iterations for MM of estimation. Default = 300. :refine r - Removes parameters whose |t| is LT r :nomm - Do not use MM starting values for unrestricted models. :nonlls - Do not use NLLS for unrestricted model. :maxbc n - Sets maximum backforecasting. Default = 20. :tolbc r - Sets convergence tolerance for backforecasting. Backcasting terminates when abs value of backcast < tolbc. Usually set as a fraction of the SD of series. default = 0.0 => tolbc=.01*series sd. :tolss r - Sets convergence tolerance. Default = 0.0. :warn - Turns on nlls warning messages. If this option is used, a common message regarding convergence can usually be ignored. :forecast n nf - Sets base and # of forecasts :foreprob r - Sets Probability limit for forecasts. Default = .95 :itprint - Prints from IMSL routines MM estimates and forecasts. This is usually not needed. :print - Print results of estimation. Variables created if options selected: %numar - Number of AR parameters %arparms - AR parameters %arorders - AR orders %numma - Number MA parameters %maparms - MA parameters %maorders - MA orders %fcast - Vector of forecasts %foreobs - Vector of Forecast obs %fconf - Forecast conf. Int %fpsi - Forecast psi weights %seriesm - Series mean %coef - constant, ar parameters, ma parameters %se - Coefficient Standard Errors %t - Coefficient t scores %cname - Coefficient name %corder - Coefficient order %const - = %seriesm*(1-par(1)-..-par(nrap) %nres - nob - (max(arorder,maorder)+2) %res - Residual vector of length %nres %resobs - Observation # of residual %y - Y vector lined up same as %res %yhat - Estimated y %avar - Random shock variance %yvar - Y variable name %rss - Residual sum of squares %sumabs - Sum of |e(t)| %maxabs - Maximum |e(t)| %yvar - Y variable Unless it fails to solve, the method of moments starting values will substantially speed up calculations. The switch :nomm can be used to turn off MM starting values if there are problems. A better choice would be to make the model simplier. To estimate an unrestricted ar(4) model call arma(gasout :nar 4 :print); To estimate an unrestricted arma(2,1) model call arma(gasout :nar 2 :nma 1 :print); To estimate a restricted model with AR terms at lag 2 and 3. call arma(gasout :arorder idint(array(:2 3)) :print) Example 1. AR model on gasin series: b34sexec options include('c:\b34slm\gas.b34'); b34srun; b34sexec matrix; * Model Discussed in Box-Jenkins and in Stokes (1997); call loaddata; call arma(gasin :nar 3 :forecast 296 24 :itprint :print); call graph(%res); call graph(%y,%yhat); call graph(acf(%res)); b34srun; Example 2. ARMA model on real m1 b34sexec options include('c:\b34slm\b34sdata.mac.b34') memb34(res79); b34srun; b34sexec matrix; call loaddata; diff2rm=dif(fmscom,2,1); call arma(diff2rm :nar 2 :nma 1 :itprint :print); call graph(%res); call graph(%y,%yhat); call graph(acf(%res)); call arma(diff2rm :nar 2 :maorder idint(array(:3,4,7)) :itprint :print); call graph(%res); call graph(%y,%yhat); call graph(acf(%res)); b34srun; AUTOBJ - Automatic Estimation of Box-Jenkins Model The ARMA command estimates univariate BJ models using ML and method of moments. Since only one AR and MA factor is allowed, this command can be used to select relatively simple models from inside a user selected framework. If many series are to be filtered quickly, this command should be considered. Models with very many terms can be estimated. The more complex command AUTOBJ will automatically identify models with AR, MA, SAR and SMA factors without the user having to specify the model. This use of time series AI allows filtering of a large number of quite different series possible. A limit of 10 terms can be in the model but up to 6 factors can be estimated. These limits are due to the Box- Jenkins philosophy that suggests parsimonious models be used. The AUTOBJ command is based on the BJIDEN and BJEST routines available as B34S commands. The underlying code is based on the Peck Box Jenkins program that was developed under the supervision of George Box at UW starting in the late 60's. In addition to automatic model selection using the :autobuild option, the AR amd MA parameters can be specified in "manual" mode of operation.. call autobj(x :options); x series to filter. If the user wants to impose differencing, this should be done outside the command or inside the command with the command :rdif or :sdif. Other wise using automatic model building, differencing will be selected if the AR parameter is above the :roottol value which defaults to .8. :autobuild - Automatically selects the arima model starting from a "generic" arima(1,1) model on appropriately differenced data. :rawacfpacf - Give Raw ACF and PACF prior to model being fit.. :difrawacf - Gives difference as well as raw acf and pacf if :rawacfpacf set. :assumptions - Lists assumptions. Not usually used. :seasonal n - Sets the seasonal period. If this is not present seasonal differencing will not be attempted. :seasonal2 n - Sets the second seasonal period. If seasonal2 is set, seasonal must be set. Used with hourly and weekly data. :longar n - Sets initial default AR order. Default=1. Range 0-3. This is not allowed if seasonal2 is set. :longma n - Sets initial default MA order. Default=1. Range 0-3. This option is not allowed if seasonal2 is set. :nodif - Suppress automatic differencing selection. :rdif - Forces Regular Differencing. :sdif - Forces Seasonal Differencing. :trend - Estimate a trend if there is differencing. :noest - No estimation will be performed. This option requires that the model has been saved. :cleanmod - On the last step, the model will be cleaned of parameters that have |t| values LT droptol. This option makes a very parsimonious model. :forcedstart - Forces a default starting value of .1 to be set. This is usually not needed. :nosearch - Turns off spike hunting. :spikelimit i - Sets limit to look for spikes. Default = max(12,2*seasonal) :spiketol r - Sets t for spike inclusion. Default = droptol. If this is set too low the program will cycle since a term will be added which will not be significant due to the |t| not meeting the droptol. :arlimit r - Sets a value to check for |t| of adjacent ACF terms. If r is set smaller, it is more likely AR terms will be added. Change this value with caution. Default = 1.3. :startvalue r - Sets default parameter start value for automatic model building. Default = .1 :print - Print results. :printres - Print residuals. :printit - Print iterations :printsteps - Prints Model selection steps for automatic model building. :backforecast - Use backforecasting. This option allows residuals to be calculated for all data points. It can result in instable estimation. This option should be used with care. :maxtry n - Maximum tries at auto model selection. Default = 4. :roottol r - Set auto model differencing tolerance. Default = .8 :droptol r - Sets drop tolerance. Default = 1.7 :eps1 r - Sets max change in relative sum of squares before iteration stops. Default = 0.0 => this criterian not used. :eps2 r - Sets relative max change in each parameter. Default = .004 :maxit i - Sets maximum number of iterations allowed. Default = 20 :nac i - Sets # autocorrelations printed. Max = 999. :npac i - Sets number of partial autocorrelations printed. :holdout n - Sets number of observations to hold out Options to override auto selection of the model. Note: Specify AR and MA in this order if present. :ar ivec - set AR orders. Can specify up to three factors. For example: :ar index(1 2 3) index(12) :ma ivec - set ma orders. Can specify up to three factors. For example: :ma index(1 2 3) index(12) :arparm rarray - Initial ar values. Usually not needed. :maparm - Initial ma values. Usually not needed. :dif ivec - set differencing orders. Can specify up to three factors. For example :dif index(1 1) :dif index(1 1) index(1 12) :dif index(1 1) index(1 12) index(1 48) Note: :dif index(1 1 1 12) not supported! :forecast index(i1 i2) - Sets forecast number and origin. Limit for number = 100 :smodeln - Sets model save name. If :noest is in effect,this sets the model name to used to make forecasts. Variables created if options selected: %numar - Number of AR factors %numma - Number MA factors %numdif - Number difference factors ********************************************** Defined if %numar > 0 %arparms - AR parameters %arse - SE of AR parameters %arord - AR orders %narfact - Number of parameters in each factor ********************************************** Defined if %numma > 0 %maparms - MA parameters %mase - SE of MA parameters %maord - MA orders %nmafact - Number of MA parameters in each factor ********************************************** Defined if %numdif > 0 %diford - Dif Orders (6 element array) ********************************************** %coef - constant, ar parameters, ma parameters %se - Coefficient Standard Errors %t - Coefficient t scores %cname - Coefficient names 123456 AR - 1 AR - 2 MA - 1 MA - 2 give info on the factor %corder - Coefficient order Defined if Forecasting ********************************************* %fcast - Vector of forecasts %foreobs - Vector of Forecast obs %fse - Forecast standard error %fpsi - Forecast psi weights %nres - nob -(max(arorder,maorder)+2) %res - Residual vector of length %nres. %resobs - Observation # of residual %np - Number of parameters %y - Y vector lined up same as %res. %yhat - Estimated y %yvar - Y variable name %rss - Residual sum of squares %rse - Residual Standard error sqrt(%rss/dfloat(%resobs-%np)) %sumabs - Sum of |e(t)| %maxabs - Maximum |e(t)| Notes: If :ar or :ma is found, auto identification will not be performed. If auto identification is used, the beginning values will often be close to the final values because of the "hidden" identification estimation runs. The switch :printsteps will show these estimations although usually this is not needed. The following statement will detect if the program ran: if(kind(%res).eq.-99)then; call print('AUTOBJ failed'); endif; Example # 1 Identify the Gas model: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(rtest); /$ /$ This roottol setting forces no differencing /$ /$ call autobj(gasout :print :nac 24 :npac 24 /$ :roottol .99 :autobuild ); /$ This turns off differencing call autobj(gasout :print :nac 24 :npac 24 :nodif :autobuild ); call rtest(%res,gasout,48); /$ Default let program decide call autobj(gasout :print :nac 24 :npac 24 /$ :printsteps :spiketol 2.0 :autobuild ); call rtest(%res,gasout,48); b34srun; Example # 2 Identify Retail Data b34sexec options ginclude('b34sdata.mac') member(retail); b34srun; b34sexec matrix; call loaddata; call load(rtest); call autobj(applance :autobuild :seasonal 12 :nac 36 :print :assumptions /$ /$ maxtry limits model /$ :printsteps :maxtry 2 /$ :forecast index(20,norows(applance)) ); call names(all); call tabulate(%cname,%corder,%coef,%se,%t); call print(%yvar,%numar,%numma,%numdif); if(%numdif.ne.0)call print(%diford); if(%numar.ne.0) call print(%narfact,%arord,%arparms,%arse); if(%numma.ne.0) call print(%nmafact,%maord,%maparms,%mase); b34srun; AUTOCOV - Autocovariance of a series call autocov(series,auto,nn); series = data auto = autocovariance nn = # of elements in auto Note: AUTOCOV is a Matrix Command subroutine contained in matrix2.mac. Before it is run it must be loaded with call load(autocov); Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call echooff; call loaddata; call load(autocov); call autocov(gasout,aa,norows(gasout)/2); aatest=acf(gasout,norows(gasout)/2); call tabulate(aa,aatest); call graph(aatest :heading 'ACF'); call graph(aa :heading 'Autocovariance'); b34srun; BACKSPACE - Backspace a unit call backspace(n); backspaces unit n Example: b34sexec matrix; /$ /$ Notes: After the call copyf unit 6 has hit an /$ end of file. /$ The call backspace(6); makes this file /$ able to be written. The call to echooff; /$ is needed since the call to rewind /$ will be echoed in the output /$ file before the backspace is given and /$ cause problems!! /$ x=rn(matrix(4,4:)); xi=inv(x); call print(x,xi); call open(77,'b34sout.out'); call rewind(77); call echooff; call copyf(6,77); call backspace(6); call echoon; b34srun; BDS BDS Nonlinearity test. call bds(x,eps1,m,dim,bdsu,bdsv,probu,probv) Calculates BDS test for series x. Eps1 should be in range .5 to 2.0. Eps used is sd*eps1. If sd = 200 and eps1 = .5 , then eps = 100. M = dimension of test. Upper limit = 17. X must be filtered such that there is no autocorrelation remaining. If this is not done, the BDS test will give a false positive reading. DIM = a vector for the dimension of the test. DIM goes from 2 .. M. BDSU = U form of test. BDSV = V form of test. PROBU = probability associated with BDSU PROBV = probability associated with BDSV. The BDS test is based on subroutines from Patterson & Ashley which in turn based their routines on LeBaron (1997). If : is added to the argument list the results will be printed. A short form of the command is call bds(x,eps1,m:); Example: b34sexec options ginclude('b34sdata.mac') member(blake); b34srun; b34sexec matrix; call loaddata; call print('Results should be:' ' 2 3 4 5 ' ' -.086613 -1.6219 -1.8737 -1.2281'); call bds(blake,.5,5,mm,bdsu,bdsv,pbdsu,pbdsv); call tabulate(mm,bdsu,bdsv,pbdsu,pbdsv); b34srun; BESTREG Best OLS REGRESSION call bestreg(gasout gasin{1 to 6} gasout{1 to 6} :print); Calculates best subset regression. Define nvar = number of right hand side variables. Options: :print Print Results including variable means. This is the usual setting. :iprint Internal print. Not usually needed. :iprint_cov Internal print. Not usually needed. :crit i => -i => use R^2 for subsets 1,...,|i| => 1 => use R^2 for subsets 1,..., nvar This is the default. => 2 => use adjusted r^2 subsets 1,..,nvar => 3 => use Mallows C(p) for subsets of 1 - nvar This switch sets %NSIZE as the number of subsets. :nbest i => # of best regressions to be found for each subset. If Adjusted R^2 or Mallows is used, then nbest overall regressions are used. Default=1 :ngood i => Maximum number of good regressions of each subset size to be saved. Must be gt nbest. Default= 10. :sample mask - Specifies a mask real*8 variable that if = 0.0 drops that observation. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better option is :holdout. :holdout n - Sets number of observations to hold out. This is useful for model validation purposes. Note: :sample option cannot be used with :holdout. Variables Created: %YVAR - Name of left hand variable. %Y - Dependent Variable Data. %NAMES - Names of exogenous variables as entered. %LAG - Lag of exogenous variable as entered. %COV => Variance Covariance Matrix. Last Col for Y %cov is %nvar,%nvar. %NVAR => Number of right hand variables plus left hand variable minus constant. %NOB => # of Observations in Model %NSIZE => Max subset considered. %NBEST => # of best regressions to be found. %NGOOD => Maximum number of good regressions of each subset to be saved. %MEANS => Means of data. Y at end. %ICRITX => Array of length nsize+1 contains locations in %crit of first elements for each subset. %CRIT => Vector of length max(%ngood*%nsize,%nvar). containing in its first %icritx(%nsize+1)-1 elements the criterion values for each subset considered. %IVARX => Index vector of size %nsize+1 containing locations in %indvar of the first element for each subset size %INDVAR => Index vector of length %ngood*%nsize*(%nsize+1)/2%ivarx(%nsize+1)-1 containing the variable numbers of each subset in same order as %crit %ICOEFX => Index vector of length %ntbest+1 containing the locations in %coef of the first row for each of the best regresions. NTBEST ICRIT < 0 -NBEST*ICRIT 1 NBEST*(NVAR-1) 2 NBEST 3 NBEST %NTBEST See above. %COEF array(ldcoef,5) containing statistics ldcoef defined as icrit < 0 -%nbest*%icrit*(1-%icrit)/2 1 %nbest*(%nvar-1)*%nvar/2 2 %nbest*(%nvar-1) 3 %nbest*(%nvar-1) saved data is: 1 Variable # 2 Coefficient Estimate 3 Estimated Standard Error 4 t-stat 5 p value for two sided test Note: This comamnd uses the IMSL routines dc2rvc and dr2est. Example One: b34sexec options ginclude('b34sdata.mac') member(draper_s_b);b34srun; b34sexec matrix; call loaddata; call olsq(y x1 x2 x3 x4 :print); call stepwise(y x1 x2 x3 x4 :print :printsteps); call bestreg (y x1 x2 x3 x4 :print); call print('Using Criterion of Adjusted R^2 ':); call print('_______________________________ ':); call bestreg (y x1 x2 x3 x4 :crit 2 :print ); call print('Using Criterion of Mallows C(p) ':); call print('_______________________________ ':); call bestreg (y x1 x2 x3 x4 :crit 3 :print ); call print('Using Criterion of Adjusted R^2 but le 3':); call print('_______________________________ ':); /; Note: :crit -3 will attampt a math operation. /; The function sfam( ) forces generation of a temp call bestreg (y x1 x2 x3 x4 :crit sfam(-3) :print); b34srun; Example Two: /; Looks at a way too big problem b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call loaddata; nlag=20; call olsq(gasout gasout{1 to nlag} gasin{1 to nlag} :print); call stepwise(gasout gasout{1 to nlag} gasin{1 to nlag} :print); call bestreg(gasout gasout{1 to nlag} gasin{1 to nlag} :print); b34srun; B_G_TEST Breusch-Godfrey (1978) Residual Test call b_g_test(iorder,x,res,gbtest,gbprob,iprint,iprint2) subroutine b_g_test(iorder,x,res,gbtest,gbprob,iprint,iprint2); /; /; Implements Breusch- Godfrey (1978) Test /; See Greene (2000) page 541 /; /; iorder => Max order of the test /; x => original x matrix /; res => residual from original equation /; gbtest => Breusch-Godfrey (1978) test Stat /; gbprob => Probability of stat /; iprint => ne 0 prints results /; iprint2 => ne 0 prints stage 2 results /; +++++++++++++++++++++++++++++++++++++++++++++++++++ /; /; Use: call olsq(y x1 x2 x3 :savex); /; do iorder=1,4; /; call b_g_test(iorder,%x,%res,gbtest.gbprob,1,0); /; enddo; /; Example: /; /; Test Case From Greene (2000) page 541 /; b34sexec options ginclude('greene4.mac') member(a13_1); b34srun; b34sexec matrix; call loaddata; call load(b_g_test); call echooff; call olsq(realnvst realgnp realint :print :savex); call print(' ':); do iorder=1,4; call B_G_test(iorder,%x,%res,gbtest,gbprob,1,0); enddo; b34srun; BGARCH Calculate function for a BGARCH model. call bgarch(res1,res2,arch1,arch2,y1,y2,func, maxlag,nbad :options); The BGARCH subroutine supports a general way to setup a BGARCH (bivariate GARCH) model and avoid the overhead of recursive calls. Fixed correlation and time varrying correlation models are possible. The BGARCH command works with two series. If more than two series are desired, use the Fortran implementation. The purpose of the BGARCH command is to provide aflexible way to input a general BGARCH model. If more complex models are desired, the best way to proceed for a recursive system is to hard code the model in Fortran. An example below shows these alternative ways to proceed. The BGARCH subroutine calculates the function which is then maximized with CMAXF2 or the in more complex cases with the nonlinear programing with nonlinear constraints command NLPMIN1. The latter approach allows nonlinear restrictions on the parameters but does not give SE's. By use of a bootstrap SE's can be obtained at substantial computer cost. BGARCH modeling in RATS often has a problem which result in the message "useable observations" that arises because during the iteration phase in the second moment equation the value goes LE 0 causing problems with the LOG and the division. If BGARCH is used with the CMAXF2 command it is possible to restrict the parameters of the second moment equation such that this does not occur. The b34s BGARCH subroutine is slower than Rats, but provides complete instrumentation of the solution process and will not give the "useable observations" message that indicates that only a reduced number of datapoints are using in estimating the model. Required BGARCH Subroutine arguments res1 - first moment residual for series 1 res2 - first moment residual for series 2 arch1 - second moment residual for series 1 arch2 - second moment residual for series 2 y1 - Series 1 y2 - Series 2 func - function maxlag - maxlag of model for purposes of ML sum. nbad - number of bad datapoints :rho - rho parameter or 3 parameters in :tvrho :tvrho - Name for rho series for tvrho. This series must be allocated and must be the same length as y1. If res1, res2, arch1 or arch2 are allocated prior to the call to BGARCH, the initial values placed in these series are used. If BGARCH allocates these series, all values are set to 0.0. Options supported :ar11 arparm arorder - AR parameters & orders series 1 for equation 1. :ar12 arparm arorder - AR parameters & orders series 2 for equation 1. :ar22 arparm arorder - AR parameters & orders series 2 for equation 2. :ar21 arparm arorder - AR parameters & orders series 1 for equation 2. :ma11 maparm maorder - MA parameters & orders series 1 for equation 1. :ma12 maparm maorder - MA parameters & orders series 2 for equation 1. :ma22 maparm maorder - MA parameters & orders series 2 for equation 2. :ma21 maparm maorder - MA parameters & orders series 1 for equation 2. :gar11 garparm garorder - GAR parameters & orders for second moment eq for series 1 to series 1. :gar21 garparm garorder - GAR parameters & orders for second moment eq for series 1 to series 2. :gar12 garparm garorder - GAR parameters & orders for second moment eq for series 2 to series 1. :gar22 garparm garorder - GAR parameters & orders for second moment eq for series 2 to series 2. :gma11 gmaparm gmaorder - GMA parameters & orders for second moment eq for series 1 to series 1. :gma21 gmaparm gmaorder - GMA parameters & orders for second moment eq for series 1 to series 2. :gma22 gmaparm gmaorder - GMA parameters & orders for second moment eq for series 2 to series 2. :gma12 gmaparm gmaorder - GMA parameters & orders for second moment eq for series 2 to series 1. :mu11 muparm muorder - Mu parameters and order for second moment eq for series 1 mapping to series 1. :mu21 muparm muorder - Mu parameters and order for second moment eq for series 1 mapping to series 2. :mu22 muparm muorder - Mu parameters and order for second moment eq for series 2 mapping to series 2. :mu12 muparm muorder - Mu parameters and order for second moment eq for series 2 mapping to series 1. :rho rhoname - Pass D1 array with one element for constant correlation model, three elements for tvrho model. :tvrho series - Pass a series name for time varrying rho. This is the rho vector. It must be allocated prior to the call. Its size is the same as data1. :negrho - The default is to assume a positive rho for the time varrying rho. The negative rho constraints the rho to be negative. While usually rho is positive, if there are convergence problems impose the negative constraint. :constant cparm - Constant. If no constant is desired, do not pass this parameter to the maximize command. cparm must be a 4 element array. It it is not present zero is assumed. :dorange irange - Sets do range. Usually this is not needed. Default is 1 to noob. Form of BGARCH Model Constant correlation case: max sum(gdet -0.5*((res1(t)**2/ arch1(t)) + (res2(t)**2/ arch2(t)) -2*rho*res1(t)*res2(t)/sqrt(arch1(t)*arch2(t))) /(1.0-rho**2)) where gdet = -0.5*(log(arch1(t))+log(arch2(t)) + log(1.0-rho**2)) Time varrying case: max sum(gdet -0.5*((res1(t)**2/ arch1(t)) + (res2(t)**2/ arch2(t)) -2*rho*res1(t)*res2(t)/ dsqrt(arch1(t)*arch2(t)))/(1.0-rho(t)**2)) rho = q0 +q1*rho(t-1) +q2*res1(t-1)*res2(t-1)/ dsqrt(arch1(t-1)*arch2(t-1)) rho = dexp(rho(t))/(1.0+dexp(rho(t)) gdet = -0.5*(log(arch1(t))+log(arch2(t)) + log(1.0-rho(t)**2)) where: res1(t)=y(t)-cparm(1) -arparm11(1)*y1(t-arorder11(1))-... -arparm12(1)*y2(t-arorder12(1))-... -maparm11(1)*res1(t-maorder11(1))-... -maparm12(1)*res2(t-maorder12(1))-... -muparm11(1)*dsqrt(arch1(t-muorder11(1)))-... -muparm12(1)*dsqrt(arch2(t-muorder12(1)))-... res2(t)=y(t)-cparm(2) -arparm22(1)*y2(t-arorder22(1))-... -arparm21(1)*y1(t-arorder21(1))-... -maparm11(1)*res1(t-maorder11(1))-... -maparm21(1)*res2(t-maorder21(1))-... -muparm21(1)*dsqrt(arch1(t-muorder21(1)))-... -muparm22(1)*dsqrt(arch2(t-muorder22(1)))-... arch1(t)=cparm(3) +gmaparm11(1)*(res1(t-gmaorder11(1))**2) + +gmaparm12(1)*(res2(t-gmaorder12(1))**2) + +garparm11(1)* arch1(garaorder11(1)) + ... +garparm12(1)* arch2(garaorder12(1)) + ... arch2(t)=cparm(4) +gmaparm22(1)*(res2(t-gmaorder22(1))**2) + +gmaparm21(1)*(res1(t-gmaorder21(1))**2) + +garparm22(1)* arch2(garaorder22(1)) + ... +garparm21(1)* arch2(garaorder12(1)) + ... Note: If overflows occur the parameters of the model may have to be restricted in such a way that they do not get near 0.0 during the solution iterations. Since the order in which the equations are solved is res1 and res2, if muorder(1)=0, then the system will have to be coded with DO loops. Examples of alternative coding are contained in BGARCH_B and BGARCH_C jobs in matrix.mac. Sample Jobs Job # 1 is a Diagonal Constant Correlation Model b34sexec scaio readsca /$ file('/usr/local/lib/b34slm/findat01.mad') file('c:\b34slm\findat01.mad') dataset(d_HKJA); b34srun; b34sexec matrix; call loaddata; count=0.0; call echooff; program test; /$ Here we have only diagonal elements zero=0.0d+00; call bgarch(res1,res2,arch1,arch2,data1,data2, func,7,nbad :ar11 array(:p6) index(6) :constant array(:zero,zero,a0,b0) :gma11 array(:a1) index(1) :gar11 array(:a2) index(1) :gma22 array(:b1) index(1) :gar22 array(:b2) index(1) :dorange index(8,469) :rho array(:rho)); count=count+1.0; /$ Optional Visual Solution info call outdouble(10,1 , func); call outdouble(10,2 , count); call outdouble(10,3, p6); call outdouble(10,4, a0); call outdouble(10,5, b0); call outdouble(10,6, a1); call outdouble(10,7, a2); call outdouble(10,8, b1); call outdouble(10,9, b2); call outdouble(10,10, rho); return; end; call print(test); j=integers(1,469); data1 = ln_hk(j); data2 = ln_ja(j); arch1 = data1*0.0 ; arch2 = data1*0.0 ; res1 = data1 ; res2 = data2 ; /$ a0 = .1, a1 = .1, a2 = .4 /$ b0 = .1, b1 = .2, b2 = .6 /$ p6 = .1, rho = 0.1 call cmaxf2(func :name test :parms p6 a0 b0 a1 a2 b1 b2 rho :ivalue array(:.1 .1 .1 .1 .4 .1 .6 .1) :maxit 300 :gradtol .1e-4 :lower array(:-.5 ,.1d-12,.1d-12,.1d-12,.1d-12, .1d-12,.1d-12,.1d-12) :upper array(:.1d+30,.1d+30,.1d+30,.1d+30,.1d+30, .1d+30,.1d+30,.1d+30) :print); b34srun; Job # 2 Is Constant Correlation but NOT diagonal /$ BGARCH Constant Correlation Example 9.2, 9.22 /$ page 369 /$ b34sexec scaio readsca /$ file('/usr/local/lib/b34slm/findat01.mad') file('c:\b34slm\findat01.mad') dataset(M_IBMLN2); b34srun; /$ /$ See Tsay(2001) page 368 Example 9.2 Equation 9.22 /$ See BGARCH_B test case /$ Uses BGARCH b34sexec matrix; call loaddata; program test; /$ /$ Rats setup info /$ /$ c1 p11 p22 p12 c2 a0 a11 b11 b12 b0 a21 a22 /$ b21 b22 rho /$ a1t=r1(t)-c1-p11*r1{1}-p22*r1{2}-p12*r2{2} /$ a2t=r2(t)-c2 /$ gvar1=a0+a11*a1t(t-1)**2+b11*h1(t-1)+ /$ b12*h2(t-1) /$ gvar2=b0+a21*a1t(t-1)**2+a22*a2t(t-1)**2+ /$ b21*h1(t-1) + b22*h2(t-1) /$ gdet=-0.5*(log(h1(t)=gvar1(t))+log(h2(t)= /$ gvar2(t))+log(1.0-rho**2)) /$ garchln = gdet-0.5/(1.0-rho**2)* /$ ((a1t(t)**2/h1(t))+(a2t(t)**2/h2(t)) /$ -2*rho*a1t(t)*a2t(t)/sqrt(h1(t)*h2(t))) call bgarch(res1,res2,arch1,arch2,data1,data2, func,3,nbad :ar11 array(:p11 p22) index(1 2) :ar12 array(:p12) index(2) :gma11 array(:a11) index(1) :gar11 array(:b11) index(1) :gar12 array(:b12) index(1) :gma22 array(:a22) index(1) :gma21 array(:a21) index(1) :gar21 array(:b21) index(1) :gar22 array(:b22) index(1) :rho array(:rho) :dorange index(3,888) :constant array(:c1 c2 a0 b0)); count=count+1.0; /$ Optional instrumentation call outdouble(10,1 , func); call outdouble(10,2 , count); call outdouble(10,3, c1); call outdouble(10,4, p11); call outdouble(10,5, p22); call outdouble(10,6, p12); call outdouble(10,7, c2); call outdouble(10,8, a0); call outdouble(10,9, a11); call outdouble(40,1, b11); call outdouble(40,2, b12); call outdouble(40,3, b0); call outdouble(40,4, a21); call outdouble(40,5, a22); call outdouble(40,6, b21); call outdouble(40,7, b22); call outdouble(40,8, rho); return; end; call print(test); /$ c1 = 1.4, c2 = 0.7, p11 = 0.1, p22 = 0.1, p12 = -0.1 /$ a0 = 3.0, a11=0.1, a21=0.02, a22=0.05 /$ b0=2.0, b11=.8, b12=.01, b21=.01, b22=.8, rho = 0.1 count=0.0; j=integers(1,888); data1=ibmln(j); data2=spln(j); call echooff; call cmaxf2(func :name test :parms c1 p11 p22 p12 c2 a0 a11 b11 b12 b0 a21 a22 b21 b22 rho :ivalue array(:1.4, .1, .1, -.1, .7 3.0, .1, .8, .01, 2.0, .02, .05,.01, .8, .1) :maxit 30000 :maxfun 30000 /$ /$ Rats Names /$ c1 p11 p22 p12 c2 /$ a0 a11 b11 b12 b0 /$ a21 a22 b21 b22 rho /$ :lower array(:.1d-12,.1d-12,.1d-12,-.2, .1d-12, .1d-12,.1d-12,.1d-12,-.06, .1d-12, .1d-12,.1d-12,-.1, .1d-12,.1d-12) :upper array(:.1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3) :print); b34srun; Job # 3 is a Time-varying Correlation Model /$ b34sexec scaio readsca /$ file('/usr/local/lib/b34slm/findat01.mad') file('c:\b34slm\findat01.mad') dataset(m_ibmln2); b34srun; /$ /$ stablemod = 1 /$ Forces GARCH parameters GE 0.0 /$ Tsay RATS code allows "unstable" models due to /$ unconstrained estimator /$ stablemod = 1 can cause problems in convergence since /$ parameters are at theirt zero point /$ /$ => stablemod=0 => -3678.3455 /$ => stablemod=1 => -3685.0870 %b34slet stablemod=0; b34sexec matrix; call loaddata; program test; call bgarch(res1,res2,arch1,arch2,ibmln,spln,func,3,nbad :ar11 array(:p1) index(1) :ar12 array(:p3) index(2) :gar11 array(:b1) index(1) :gma11 array(:a1) index(1) :gar12 array(:f1) index(1) :gma22 array(:a11) index(1) :gar22 array(:b11) index(1) :gma21 array(:d11) index(1) :gar21 array(:f11) index(1) :rho array(:q0,q1,q2) :tvrho rho :dorange index(3,888) :constant array(:c1 c2 a0 a00)); count=count+1.0; /$ Optional visual output to monitor solution progress call outdouble(10,1 , func); call outdouble(10,2 , count); call outdouble(10,3, c1); call outdouble(10,4, p1); call outdouble(10,5, p3); call outdouble(10,6, c2); call outdouble(10,7, a0); call outdouble(10,8, a1); call outdouble(10,9, b1); call outdouble(40,1, f1); call outdouble(40,2, a00); call outdouble(40,3, a11); call outdouble(40,4, b11); call outdouble(40,5, f11); call outdouble(40,6, d11); call outdouble(40,7, q0); call outdouble(40,8, q1); call outdouble(40,9, q2); /$ /$ Trap > 0 value and reset /$ if(func.gt.0.0)func=-10.d+9; return; end; call print(test); count=0.0; /$ c1 = 1.4, p1 = 0.1, p3 =-.1 , c2 = .07, a0 = 2.95 /$ a1 = .08 b1 = .87 f1 =-.03 a00= 2.05 a11=.05 /$ b11= .92 f11=-.06 d11=.04 q0 = -2.0 q1 = 3.0 /$ q2 = .1 j=integers(888) ; data1=ibmln(j) ; data2=spln(j) ; res1 =array(norows(data1):); res2 =array(norows(data1):); arch1=array(norows(data1):) + 45.; arch2=array(norows(data2):) + 31.; rho =array(norows(data2):) + .8 ; call echooff; call cmaxf2(func :name test :parms c1 p1 p3 c2 a0 a1 b1 f1 a00 a11 b11 f11 d11 q0 q1 q2 /$ /$ c1, p1, p3, c2, a0 /$ a1, b1, f1, a00, a11 /$ b11 ,f11, d11, q0, q1 /$ q2 /$ Rats Answers reported in Tsay (2001) /$ Note that Tsay allows GARCH parameters to be < 0 !!!! /$ This make problem unstable!!!!! /$ See f1 & f11 /$ /$ 1.3178 .076103 -.068349 .673403 2.79865 /$ .08364 .8642 -.01995 1.7101 .05401 /$ .9139 -.05811 .03711 -2.0239 3.983 /$ .08755 /$ Rats input values /$ :ivalue array(: 1.4, 0.1, -.1 , .07, 2.95, /$ .08, .87, -.03, 2.05, .05, /$ .92, -.06, .04, -2.0, 3.0, /$ .1) /$ Good Values that are close to rats input values except /$ for GARCH parameters which are not allowed to go < 0.0 %b34sif(&stablemod.eq.1)%then; :ivalue array(: 1.4, .08, -.07, .7, 2.95, .08, .87, .01, 2.05, .05, .92, .01, .04, -2.0, 3.0, .1) :lower array(:.1d-12, .1d-12, -.4, .1d-12, .1d-12, .1d-12, .1d-12, .1d-12 .1d-12, .1d-12, .1d-12, .1d-12, .1d-12, -6., .1d-12 .1d-12) %b34sendif; %b34sif(&stablemod.eq.0)%then; /$ /$ These values "beat Tsay" but model is not stable!!! /$ Stokes feels that GARCH mdoels should be estimated /$ with constraints suggested by theory!! This is not /$ possible with older versions of RATS /$ :ivalue array(: 1.3, .08, -.07, .7, 2.8 , .08, .87, -.01, 1.7 , .05, .91, -.01, .04, -2.0, 4.1, .08) /$ These values suggested by Tsay cause problems /$ :ivalue array(: 1.4, 0.1, -.1 , .07, 2.95, /$ .08, .87, -.03, 2.05, .05, /$ .92, -.06, .04, -2.0, 3.0, /$ .1) :lower array(:.1d-12, .1d-12, -.4, .1d-12, .1d-12, .1d-12, .1d-12, -.02 .1d-12, .1d-12, .1d-12, -.06 .1d-12, -4., .1d-12 .1d-12) %b34sendif; :upper array(:.1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+3, .1d+1, .1d+3 .1d+3) :maxit 30000 :maxfun 30000 :maxg 10000 :print); b34srun; Notes: The jobs BGARCH_A, BGARCH_B and BGARCH_C show BGARCH, Fortran and RATS jobs for the same problem. These jobs show exactly what is being estimated. For further detail, see Tsay (2002) whose data we use and who developed these models. BJ_IDEN - ACF and PACF Listing and or Plotting call bj_iden(x,nacf,cc,iplot,file); Will list ACF and optionally PACF in a table suitable for analysis. Optionally will plot and save plots in file. subroutine bj_iden(x,nacf,heading,iplot,file); /; /; x => Series /; nacf => # acf/pacf calculated printed /; heading => Heading /; iplot => NE 0 => plot /; file => File to save high res plot /; Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(bj_iden); call echooff; call bj_iden(gasout,220,'Series => gasout',1,'test.wmf'); b34srun; BLUS BLUS Residual Analysis call blus(itype,x,olse,ibase,bluse,bluse2,eigb, sumeig,sumsqb,olsbeta,blusbeta,ibad,x1,teststat,iprint); Will perform BLUS Analysis on the OLS residuals. It is assumed that the data has been sorted against series x1. The BLUS subroutine uses routines BLUSBASE, BLUSTEST and BLUSRES. Usually all that is required is to call BLUS. Help files for all 4 routines are shown. This routine does not have a right side limit of 20 variables like the "historical" Theil inspired BLUS code that is under the regression command. The command call load(blus); will load all routines. subroutine blus(itype,x,olse,ibase,bluse,bluse2,eigb, sumeig,sumsqb,olsbeta,blusbeta,ibad,x1,teststat,iprint); /$ /$ Routine to calculate BLUS residuals tests /$ Routine mimics what is available in RA card /$ in regresion command. /$ /$ Routines BLUSBASE BLUSTEST and BLUSRES are needed. /$ /$ Routine built 25 June 2003 by Houston H. Stokes /$ /$ itype = 0 DW test /$ itype = 1 MVN test /$ itype = 2 Het Base /$ itype = 3 Parabola base /$ x = n by k x matrix. OLSQ command saves /$ this if :savex is effect (input) /$ olse = OLS Error. usually %res (input) /$ ibase = Integer vector of the base for the BLUS /$ calculation /$ bluse = BLUS residual vector /$ bluse2 = BLUS residual vector with base marked /$ as missing /$ eigb = eigenvalues from blus /$ sumeig = sum dsqrt of eigenvalues /$ sumsqb = sum of blus residuals squared /$ olsbeta = OLS Beta (input) /$ blusbeta= BLUS Beta /$ ibad = 0 all ok, = 1 base singular, /$ = 2 error on input /$ x1 = vector used for the sort. /$ Needed if itype = 3 /$ teststat= test statistic /$ iprint = 1 print results; subroutine blusbase(iopt,itype,n,k,ibase,nbase); /$ /$ Gets BLUS base /$ /$ iopt = 0 get number of bases for itype in nbase /$ iopt = 1 get base number nbase for itype /$ /$ Example: if N = 20 and k = 4 there are 5 bases /$ [1 2 3 4] [1 2 3 20] [1 2 19 20] /$ [1 18 19 20] [17 18 19 20] /$ itype = 0 DW and MVN base /$ itype = 1 DW and MVN base /$ itype = 2 Het Base /$ itype = 3 Parabola base /$ n = # of observations /$ k = # right hand side variables /$ ibase = Blus base /$ nbase = # of bases if iopt=0, bane subroutine blustest(bluse,x,ibase,itype,test); /$ /$ Construct Tests on BLUS Residuals /$ /$ Routine built 15 May 2003 /$ /$ bluse = Blus residuals /$ x = Vector used for the sort. Needed if /$ itype=3 /$ ibase = BLUS base integer*4 vector of k elements /$ itype = 0 DW /$ = 1 mvn /$ = 2 F /$ = 3 parabola /$ test = test value /$ call blusres(x,olse,ibase,bluse,bluse2,eigb, sumeig,sumsqb,olsbeta,blusbeta,ibad); /$ /$ Routine to calculate BLUS residuals /$ Routine mimics what is available in RA card /$ and BLUS capability in regresion command. /$ x = n by k x matrix. OLSQ command saves /$ this if :savex is effect (input) /$ olse = OLS Error. usually %res (input) /$ ibase = Integer vector of the base for /$ the BLUS calculation /$ bluse = BLUS residual vector /$ bluse2 = BLUS residual vector with base marked as missing /$ eigb = eigenvalues from blus /$ sumeig = sum dsqrt of eigenvalues /$ sumsqb = sum of blus residuals squared /$ olsbeta = OLS Beta (input) /$ blusbeta= BLUS Beta /$ ibad = 0 all ok, = 1 base singular, /$ = 2 error on input /$ /$ ********************************************** /$ Example of old and newer way to get same answers: b34sexec data heading('Theil(1971) Table 5.1'); * For detail see pages 214-216; * Matrix Command shows BLUS Calculation; * Code discussed in Stokes ( ) 3rd Edition ; build x1,x2, y; gen x1=kount(); gen x2=dsin(x1/2.0); gen y =x1+ 10.0*dsin(x1/2.)+act_e; input act_e; datacards; 1.046 -.508 -1.630 -.146 -.105 -.357 -1.384 .360 -.992 -.116 -1.698 -1.339 1.827 -.959 .424 .969 -1.141 -1.041 1.041 .535 b34sreturn; b34srun; /$ b34sexec list; b34srun; b34sexec regression residualp blus=both noint; comment('Illustrates BLUS analysis with Theil Data'); model y=x1 x2; ra resid=allblus vars(x1); b34srun; b34sexec matrix; call loaddata; call load(blus); program fulltest; iprint=1; call olsq(y x1 x2 :noint :print :savex); do itype=0,3; call blus(itype,%x,%res,ibase,bluse,bluse2,eigb,sumeig, sumsqb, %coef,blusbeta,ibad,x1,teststat,iprint); enddo; return; end; /$ call echoon; call echooff; call fulltest; b34srun; BPFILTER Baxter-King Filter. call bpfilter(data,datat,datadev,highfreq, lowfreq,nterms :); Uses Baxter-King MA band-pass filter to decompose data into trend (datat) and deviations from trend (datadev). Data must be real*8. If data is a matrix or 2d array, each column is transformed. data = real*8 data datat = trend datadev = deviations from trend highfreq = real*8 var highest freq to pass. Note: highfreq set in periods. lowfreq = real*8 var lowest freq to pass. Note: lowfreq set in periods. nterms = integer var set to number of terms in filter Note: If series in data is N observations, nterms at the beginning and end of series are set = 0. The optional paremeter : = will set these to missing. Example 1: call bpfilter(x,tx,devx,6,32,20); Example 2: /$ Illustrates passing gasout through Baxter-King MA /$ filter. goodrow and catcol used to line up data /$ for plots and redefine the series !! /$ b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; highfreq=6.; lowfreq=32.; nterms=20; call bpfilter(gasout,tr,dev,highfreq,lowfreq,nterms:); call tabulate(gasout,tr,dev,); x=goodrow(catcol(gasout,tr,dev)); gasout=x(,1); tr =x(,2); dev =x(,3); call tabulate(gasout,tr,dev); call graph(gasout,tr,dev); b34srun; The bpfilter command "hard wires" the bpf subroutine. BREAK Set User Program Break Point. call break; If any key has been hit, and this command is executed, the program will terminate after a question has been asked to confirm. Alternative call: call break('We are at point A now'); Message can be up to 40 characters. BUILDLAG Builds NEWY and NEWX for VAR Modeling call buildlag(x,nlag,ibegin,iend,newx,newy); This routine builds lags of x for VAR modeling new y is also built. x(n,k) - n,k matrix of data values nlag - Number of lags ibegin - Begin Data point iend - End Data Point newx - nob,(nlags*k) matrix of x variables newy - nob,k matrix of left hand variables nob=iend-ibegin+1-nlag Example: b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call loaddata; call load(buildlag); x=catcol(gasin,gasout); nlag=2; ibegin=1; iend=10; call print(x); call buildlag(x,nlag,ibegin,iend,newx,newy); call print(newx,newy); b34srun; CANCORR Calculate Canonical Correlations call cancorr(cc,x,z,a,lamda); will calculate the canonical correlations between X and Z. The canonical correlations are the square root of the real part of the eigen values cc1=real( eig(inv(xpx)*transpose(zpx)*inv(zpz)*zpx,a )); cc =real( eig(inv(zpz)*zpx*inv(xpx)*transpose(zpx),lamda)); cc1=sqrt(cc1); cc=sqrt(cc); a=real(a); lamda=real(lamda); subject to the constraint that X*a and Z*lamda are orthagonal. For documentation see Hall-Rudebusch-Wilcox (1996, 287) Example: b34sexec options ginclude('b34sdata.mac') member(kmenta); b34srun; b34sexec matrix; call loaddata; call echooff; call load(cancorr); x=mfam(catcol(p,d,constant)); z=mfam(catcol(d,f,a,constant)); call cancorr(cc,x,z,a,lamda); call print( cc,x,z,a,lamda); test1=x*a; test2=z*lamda; call print('x*a and z*lamda':); call print(transpose(test1)*test1); call print(transpose(test2)*test2); b34srun; == CATS Cointegration Test using Johansen Method subroutine cats(y,maxlag,eigval,eigvec,lamtrace,lammax, alpha,beta,pi,icon,itrend,iprint,ieprint); /; /; Implements Johansen (1988, ,1992) Cointegration Testing /; Methods as discussed in William Greene 2008, pp 761-767 /; /; See also Russell Davidson and James MacKinnon /; 'Econometric Theory and Methods' (2004) 640-641. /; /; This routine is experimental. It is intended as a first /; pass at testing for cointegration using the Johansen /; method. For more extended analysis, see "CATS in Rats" /; which was developed under Johansen and is available /; from Estima. /; /; y = basic data of VAR model to be tested /; maxlag = must be ge 1 /; eigval = ordered eigenvalues /; eigvec = ordered eigenvectors /; lamtrace = eigenvalue trace test /; lammax = eigenvalue max test /; alpha = Speed of adjustment matrix /; beta = BIG_Z=psi(dd)*eigvec /; psi(dd)*transpose(psi(dd)=inv(sigma(dd)) /; beta is first k by k matrix in big_Z /; pi = alpha * transpose(beta) /; icon = 1 => have constant in model /; itrend = 1 => have trend in the model /; testtab = Test table /; iprint = 1 => print results /; ieprint = 1 => 0 to print steps /; /; Basic code implemented in Beta release September 2009 /; by Houston H. Stokes /; /; Row 1 95% of Table C.1 trace test of CATS Manual /; Row 2 95% of Table C.3 of Cats Manual /; Rom 3 ? /; Row 4 95% of Table C.1 cointegrating rank Cats manual * testtab=array(4,12: 4.07 12.28 24.21 40.10 59.96 83.82 111.68 143.53 179.38 219.23 263.09 310.94 3.84 15.41 29.80 47.71 69.61 95.51 125.42 159.32 197.22 239.12 285.02 334.92 3.84 18.15 34.56 54.11 77.79 104.76 135.75 170.15 208.53 250.53 296.02 345.27 9.14 20.16 35.07 53.94 76.81 103.68 134.54 169.41 208.27 251.13 297.99 348.85); * testtab=transpose(testtab); Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(cats); call echooff; /; call cats(y,maxlag,eigval,eigvec,lamtrace,lammax, /; alpha,beta,pi,icon,itrend,iprint,ieprint); /; /; Implements Johansen (1988, ,1992) Cointegration Testing /; Methods as discussed in William Greene 2008, pp 761-767 /; /; See also Russell Davidson and James MacKinnon /; 'Econometric Theory and Methods' (2004) 640-641. /; /; Implemented September 2009 by Houston H. Stokes /; big_x =catcol(y,z,w ); nn=20000; x1=cusum(rn(array(nn:))); x2=cusum(rn(array(nn:))); x3=cusum(rn(array(nn:))); x3=5.0*x1 + .3*x2 +rn(x3)/100.; xx1=rn(array(nn:)); xx2=rn(array(nn:)); xx3=rn(array(nn:)); /; big_x=mfam(catcol(x1,x2,x3)); /; big_x=mfam(catcol(xx1,xx2,xx3)); iprint =1; maxlag =1; icon =1; itrend =1; ieprint =0; /;call cats(big_x,maxlag,eigval,eigvec,ltrace,lmax,iprint, /; icon,itrend,idebug); /; one series setup/ /; big_x=matrix(100,1:y); do maxlag=1,2; call cats(big_x,maxlag,eigval,eigvec,ltrace,lmax, alpha,beta,pi,0,0,iprint,ieprint); call cats(big_x,maxlag,eigval,eigvec,ltrace,lmax, alpha,beta,pi,1,0,iprint,ieprint); call cats(big_x,maxlag,eigval,eigvec,ltrace,lmax, alpha,beta,pi,0,1,iprint,ieprint); call cats(big_x,maxlag,eigval,eigvec,ltrace,lmax, alpha,beta,pi,1,1,iprint,ieprint); enddo; call print(' ':); call print('dif(y))':); call describe(dif(y,1,1) :print); call print(' ':); call print('dif(z))':); call describe(dif(z,1,1) :print); call print(' ':); call print('dif(w))':); call describe(dif(w,1,1) :print); b34srun; CCFTEST - Display CCF Function of Prewhitened data call ccftest(res1,y,nccf,lags,title); res1 => First Moment Residual y => Input Series nccf => Number ccf terms lags => lags title => title High resolution graphs are made. List of CCF values also given. ccftest is a subroutine and must be loaded with the statement call load(ccftest); Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(ccftest); nn=norows(gasout)/4; call character(title,'Gasin vs Gasout'); call ccftest(gasin,gasout,nn,lags,title); b34srun; CFREQ Determine Cumulative Frequency Distribution call cfreq(series,cseries,cc); series = Input series cseries = Sorted input series cc = Cumulative frequency Note: CFREQ is a Matrix Command subroutine contained in matrix2.mac. It must be loaded with call load(cfreq); before it is used. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(cfreq); call cfreq(gasout,sgasout,cc); call tabulate(gasout,sgasout,cc); b34srun; See test cases cfreq and quantile in matrix.mac CHAR1 Place a string in a character*1 array. call char1(cc,'text'); Places text in ' ' in a character literal object cc. There is currently no limit on the number of cols of text inside ' '. This text can be placed in an array. For example: call char1(c,'ABCDEFGHI'); cc=array(3,3:c); call print(cc); results in ABC DEF GHI call character( ) can be used in place of call char1. If cc is a character*8 variable of any length, the statement call char1(newc,cc); places cc in newc, a character*1 object. The variable cc can be > 72 characters. If more than two arguments are supplied, a 2D array is created where the length is automatically padded. Advanced options: Note that the B34S matrix command converts any string of length LE 8 to a character*8 object. The command call char1(cc,'a'); places the character a in a 1 element character*1 array (kind=5) with norows=1 nocols=1 since 'a' became a temp with elements 'a ' and the command automatically looked for the actual length. If the alternative setup call char1(cc,'a':1); was used, cc would be a character*1 scaler (kind=0) object containing that one element. The call call char1(cc,'a':2); would create a 2 element character*1 object. If the actual length of the input string is > 8, its actual length is used. The statement call character(test,' '); will set blanks in a kind=5 character*1 array. The statement call character(test2,' ':1); will set a blank in a klass=0 character*1 scaler. The below listed code will document this: b34sexec matrix; call print('Call 1 creates klass=0 character*1 scaler':); call print('Call 2 creates klass=5 character*1 array':); call character(t, ' ':1); call character(tt,' '); call names(all); call print(t,tt); b34srun; The statement t3=c1array(1: ); is the same as call character(t3,' ':1); Warning t3=c1array(1:' '); fails since ' ' seen as real*8 by parser. Many commands such as call csv produce a character*8 array %series. The statements call character(cc,%series); call tabulate(argument(cc)); may fail if a name is in %series of 8 characters. Blanks can be inserted into cc by the commands call character(cc,%series :9); call tabulate(argument(cc)); Examples of structured use of character arrays: b34sexec matrix; call character(cc,'1234567890qwertyuiop'); i=integers(7); ccc=cc(i); i=i+3; cccp3=cc(i); call print(cc,ccc,cccp3); * get a large character array; call character(clarge,rtoch(array(1000:))); call names(all); b34srun; b34sexec matrix; call char1(c1,'This is a long string what do you think'); call char1(c2,'This is '); call print(c1,c2); call char1(x ,'This is a long string what do you think' 'so it this ' 'But this is not'); call names(all); call print(c1,c2,x); b34srun; b34sexec matrix; /$ /$ Job shows creating char*8 and char*1 variables /$ and moving data between the variable types /$ call character(cc_3, '012'); call character(cc, '012':3); call character(cc0, '0' :1); call character(cc1, '1' :1); call names(all); call print(cc(2),cc0); if(cc(2).eq.cc0)call print('yes-error'); call print(cc(1),cc0); if(cc(1).eq.cc0)call print('yes-right1'); call print(cc(2),cc0); if(cc(2).ne.cc0)call print('yes-right2'); call print(cc(1),cc1); if(cc(1).ne.cc1)call print('yes-right3'); cc=array(:0.,1.,2.); call print(cc); if(cc(2).eq.0.)call print('yes-error'); if(cc(1).eq.0.)call print('yes-right1'); if(cc(2).ne.0.)call print('yes-right2'); if(cc(1).ne.1.)call print('yes-right3'); b34srun; Padding a character*8 array of names /; Writes with names and header but adds lines only b34sexec matrix; n=10; k=3; x=rn(array(n,k:)); call csv(:writefile 'test.csv' :var x); x=x*10.; call csv(:writefile 'test.csv' :var x :add :nonames :nodatestamp); b34srun; /; Writes without names b34sexec matrix; n=10; k=3; x=rn(array(n,k:)); call csv(:writefile 'test2.csv' :var x :nodatestamp :nonames ); x=x*10.; call csv(:writefile 'test2.csv' :var x :add :nonames :nodatestamp); b34srun; /; Test reads b34sexec matrix; call csv(:readfile 'test.csv'); call names(all); /; /; Two ways to print /; do i=1,norows(%series); call print(argument(%series(i))); enddo; /; /; Note use of padding to put a blank after the name /; call character(cc,%series :9); call print(cc); call tabulate(argument(cc)); b34srun; /; Reading a noname file using call read b34sexec matrix; n=20; k=3; call open(71,'test2.csv'); tdata=array(k,n:); call read(tdata,71); call close(71); tdata=transpose(tdata); call print(tdata); b34srun; CHARACTER Place a string in a character*1 array. call character(cc,'text'); Places text in ' ' in a character literal object cc. For detail on this command see the help file for call char1. CHTOHEX Convert a character to a hex value call chtohex(ch,hex); ch = Character*1 character vector of size n hex= character*1 character matrix of size 2*n Extended example b34sexec matrix; /$ Looking at Printable Characters ; i=integers(33,127); call igetchari(i,cc); call names(all); call tabulate(i,cc); call igetichar(cc,iitest); call chtohex(cc,hexcc); /$ Repack character*2 array save as character*1; /$ Next two statments work the same /$ hexcc2= array(norows(hexcc)/2,2:hexcc); hexcc2=c1array(norows(hexcc)/2,2:hexcc); hex1=hexcc2(,1); hex2=hexcc2(,2); call hextoch(hexcc,cctest); xx=transpose(hexcc2); call print(xx,hexcc2); call hextoch(xx,cctest2); call names(all); /$ get hexcc2 in a printable variable; blank=c1array(norows(hex1):); call names(all); c8var=catcol(hex1, hex2,blank,blank, blank, blank,blank,blank); call names(all); /$ call print(c8var); c8var=c8array(norows(c8var):transpose(c8var)); call tabulate(i,cc,iitest,hex1,hex2, cctest,cctest2,c8var); b34srun; CHECKPOINT Save workspace in portable file. call checkpoint; Will save the workspace with a default name. Alternative options can be passed with :keywords. The checkpoint command works the same as the save command except that it automatically uses the :speakeasy option. Keywords supported include: :file - to pass a file name. Default name is 'matrix.psv'. :var - to restrict saving to a list of variables. Do not place , between names. If variable is known at the local and global level, the local copy is saved. This means that formula results, not formulas are saved. If :var is not present, all objects will be saved. :speakeasy - Only pass data, no programs. If this option is used, the save file can be read by the Speakeasy(r) program. The call checkpoint; command automatically assumes this option. As a result real*16 and complex*32 variables are saved as real*8 and complex*16 respectively. If call save; is used, then this conversion is not made. While real*16 and complex*32 variables are preserved, this save file will not work with Speakeasy! :ndigits4 - Sets save format e12.4 :ndigits8 - Sets save format e16.8. :ndigits16 - Sets save format e24.16. This is the default. :ndigits32 - Sets save format e40.32. If real*16 data is to be saved, it is highly recommended that this option be used to preserve accuracy. Examples: call checkpoint(:var x y z); call checkpoint(:var x y z :file 'myrun.psv'); call checkpoint(:file 'myrun.psv'); call checkpoint(:var x y :file 'mygood.psv' ); If you are running with Speakeasy, it is suggested that you use the ending *.psv. The SAVE and RESTORE commands use a subset of the Speakeasy EXPORTALL & IMPORTALL format and are designed to facilitate moving objects from one system to another. Since B34S MATRIX programs, subroutines and functions will not work on Speakeasy, the keyword :speakeasy MUST be used to save into a file that will be read by Speakeasy(r). Since Speakeasy does not at present support real*16 and complex*32, these data types are automatically saved as real*8 and complex*16 respectively. VPA data can not be directly saved in a savefile. However VPA data can be hidden in a real*8 variable so VPA numbers can be saved with checkpoints etc using the command call vpaset(vpa r8 :saveasr8); The variable r8 can be reloaded into a VPA variable with call vpaset(r8 vpa :saveasvpa); The first four elements give kind, nr8, norows, nocols. For related commands see restore and save. CLEARALL Clears all objects from workspace. call clearall; Clears data, programs,subroutines and functions. Use with caution! See related, and safer command, cleardata. CLEARDAT Clears data from workspace. call cleardat; Clears all data. Use with caution. See related, and more dangerous command clearall. CLOSE Close a logical unit. call close(n); Closes unit n. Example: call close(72); CLS Clear screen. call cls; Clears the screen. Alternatives: call cls(arg); arg > 0 => clear row. arg LE 0 => clear window. Examples: call cls(2); clears row 2 in current window. call cls(-1); clears current window. CLUSTER K-Means and Hierarchical Cluster Models call cluster(x1 x2 :k_mean 3 :print); call cluster(x1 x2 :h_cluster 3 :print); The cluster command implements IMSL code for k-means (dkmean) and hierarchical clustering (cnumb, dtreep, dcdist dclink). Required arguments are :k_mean or :h_cluster and the data vectors. In place of data vectors, a matrix can be input. Column means are not displayed in this case. For matrix input the options :sample, :holdout, :savex are not supported. Required either :k_mean or :h_cluster. :k_mean k => Sets number of clusters for k_mean method :h_cluster k => Sets number of clusters for h_cluster analysis. Options: :print => Print results :savex => Saves the X matrix in %X. :sample mask - Specifies a mask real*8 variable that if = 0.0 drops that observation. The :sample option cannot be used if the :outlier or :rr options are used. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better option is :holdout. :holdout n - Sets number of observations to hold out. This is useful for model validation purposes. :savemodel - Saves estimated model in object %smod unless reset with :modname. Options for :k_mean :maxit m - Sets the maximum iterations. Default = 30. :clusters b - Sets the k by nvar matrix of cluster seeds. Default = x(1,), x(kk+1,),x(2*xx+1,) where kk=nob/k :weight - Specifies that the last column of the input matrix contains weights. :freq - Speifies that the last column of the input matrix contains frequency data. Note: if BOTH :weight and :freq are set the freq column must be the next to last column. Options for :h_cluster :method i 0 => single linkage (minimum distance method). Default. 1 => complete linkage (maximum distance) 2 => average distance between objects within the merged cluster. 3 => average distance between objects in the two clusters. 4 => Ward's method that minimizes the within-cluster sums of squares. For Ward's method the elements of %dist are assumed to be Euclidean distances. :dist i 0 => Euclidean distance (L-2 norm) 1 => Sum of absolute differences (L-1 norm) 2 => Maximum difference (L-infinity norm) 3 => Mahalanobis distance 4 => Absolute value of the cosine of the angle between the vectors 5 => Angle in radians (0, pi) between the lines through the origin defined by the vectors 6 => Correlation coefficient 7 => Absolute value of the correlation coefficient 8 => Number of exact matches Note: use methods > 4 with caution. These are in beta form. :htree => display a horizontal tree :hitree => display a horizontal i-tree :vtree => display a vertical tree. :iroot i => Specify the root node of the subtree to be printed. :nscale i => Specify the number of horizontal slices of the tree to be printed. :nfill i => Specify the number of filler lines printed between horizontal or vertical node lines. :scale array(r1 r2) => Vector of length two giving the interval of the on the clevel axis which should be used to plot the tree. scale(1) is the location for printing the terminal nodes. The root node is printed at scale(2) Note: since :method and :dist have integer arguments, this facilitates sensitivity investigations. :transdist i 0 => No transformation performed. Default. 1 => Convert similarities to distances by multiplication by -1. 2 => Convert similarities (usually correlations) to distances by taking the reciprocal of the absolute value. This option requires input of a distance matrix using :inputdist. :inputdist d Input distance for use with :transdist 1 and :transdist 2. Variables created: %iclus => Integer*4 vector of nob elements indicating which class each observation is assigned. %nclus => integer*4 vector of size k indicating how many observations are in each class. For :h_cluster %dist => Distance or simularity matrix %clevel => real*8 vector of nob-1 elements indicating the level at which the clusters are joined. %clson => an integer*4 vector of length N-1 containing the right son cluster numbers. %crson => an integer*4 vector of length N-1 containing the left son cluster numbers. Cluster N+I is formed by merging clusters %clson(i) and %crson(i). For :k_mean %sumw => iclass by nob matrix containing sum of weights used to compute each cluster. %wss => vector of length iclass containing within sum of squares. %tss => sum(%wss). Hastie-Tibshirani-Friedman (2009, 519) suggest using %tss for various k settings to determine the appropriate number of classes. %ave_wss => vector of length iclass containing within sum of squares divided %nclus. %clust_m => k by nvar matrix containing the cluster means. Usage notes: The :h_cluster option requires that the number of observations be greater than the number of variables. This restriction is not imposed for :k_mean option. Examples: %b34slet runr=0; %b34slet runsas=0; b34sexec options ginclude('b34sdata.mac') member(cluster); b34srun; b34sexec matrix; call loaddata; /; call olsq(y x1 x2 x3 x4 x5 x6 x7 x8 :print); /; call ranforest(y x1 x2 x3 x4 x5 x6 x7 x8 :reg :print); /; call ppreg(y x1 x2 x3 x4 x5 x6 x7 x8 :reg :print); centdat=matrix(4,8: 16.750 43.250 37.500 29.750 21.500 29.250 30.250 9.2500 27.222 31.111 47.444 62.167 13.722 21.889 38.889 10.444 31.500 29.500 51.000 82.000 17.500 16.000 14.500 8.5000 27.667 22.833 47.000 84.667 8.0000 13.167 41.500 12.667); call names(all); call cluster(x1 x2 x3 x4 x5 x6 x7 x8 :k_mean 4 :clusters centdat :print); call print(%iclus %sumw); call cluster(x1 x2 x3 x4 x5 x6 x7 x8 :h_cluster 4 :print); call print(%clevel %clson %crson /; %dist %iclus); b34srun; /; /; Data is in its own file /; Running R from a fixed script /; Uses r.bat file /; /; Unit 28 is the data %b34sif(&runr.ne.0)%then; b34sexec options open('rjob2.r') unit(28) disp=unknown$ b34srun$ b34sexec options clean(28) $ b34seend$ b34sexec options open('rjob.r') unit(29) disp=unknown$ b34srun$ b34sexec options clean(29) $ b34seend$ b34sexec pgmcall; r; pgmcards; # windows() windows(record=T) source('rjob2.r') library(cluster) library(mgcv) require(graphics) x1 b=lm(y~x1+x2+x3+x4+x5+x6) summary(b) model<-kmeans(x1+x2+x3+x4+x5+x6+x7+x8,4) model names(model) model$cluster model$centers model$withinss model$size # plot(model) model3<-gam(y~x1+x2+x3+x4+x5+x6+x7+x8) #model3<-gam(y~s(x1)+s(x2)+s(x3)+s(x4)+s(x5)+s(x6)+s(x7)+s(x8)) summary(model3) # plot(model3) model2<-hclust(dist(x1+x2+x3+x4+x5+x6+x7+x8)) model2$merge model2$height model2$order plot(model2) # demo(graphics) # quit() b34sreturn$ b34srun $ b34sexec options close(28)$ b34srun$ b34sexec options close(29)$ b34srun$ b34sexec options dodos(' r rjob' ) unix( ' R rjob') $ b34srun$ b34sexec options npageout noheader writeout(' ','output from r',' ',' ') copyfout('rjob.out') /; dodos('erase rjob.r','erase rjob.r.Rout','erase rjob2.r') $ b34srun$ %b34sendif; %b34sif(&runsas.ne.0)%then; b34sexec options header$ b34srun$ b34sexec options open('testsas.sas') unit(29) disp=unknown$ b34srun$ b34sexec options clean(29) $ b34seend$ b34sexec pgmcall idata=29 icntrl=29$ sas $ * sas commands next ; pgmcards$ proc fastclus maxclusters=4; var x1 x2 x3 x4 x5 x6 x7 x8; run; proc cluster method=ward outtree=tree; var x1 x2 x3 x4 x5 x6 x7 x8; run; proc tree; run; b34sreturn$ b34srun $ b34sexec options close(29)$ b34srun$ /$ the next card has to be modified to point to sas location /$ be sure and wait until sas gets done before b34s resumes /$ ************************************************************ b34sexec options dodos('start /w /r sas testsas' ) dounix('sas testsas' ) $ b34srun$ b34sexec options npageout noheader writeout(' ','output from sas',' ',' ') writelog(' ','output from sas',' ',' ') copyfout('testsas.lst') copyflog('testsas.log') dodos('erase testsas.sas','erase testsas.lst', 'erase testsas.log') dounix('rm testsas.sas','rm testsas.lst','rm testsas.log') $ b34srun$ b34sexec options header$ b34srun$ %b34sendif; CLUSTER2 IMSL benchmark test cases b34sexec options ginclude('b34sdata.mac') member(iris2); b34srun; b34sexec matrix; call loaddata; cc=matrix(3,4: 5.006 3.428 1.462 .246 5.902 2.748 4.394 1.434 6.850 3.074 5.742 2.071); call cluster(x1 x2 x3 x4 :k_mean 3 :print /; :clusters cc :savex); /; call print(cc,%x); call print(%iclus %sumw); call print('++++++++++++++++++++++++++++++++++':); call cluster(x1 x2 x3 x4 :h_cluster 5 :print); call print(%clevel %clson %crson /; %dist %iclus); b34srun; EXAMPLE of DMA Microarray Data b34sexec options ginclude('h_t_f_data.mac') member(cancer); b34srun; b34sexec matrix; call loaddata; call echooff; call names(:); i=norows(%names%); nn=%names%(integers(2,i-2)); /; call print(label(argument(nn(1)))); tt=label(argument(nn)); /; call print(transpose(tt)); tt=transpose(tt); bigx = catcol(x_1 x_2 x_3 x_4 x_5 x_6 x_7 x_8 x_9 x_10 x_11 x_12 x_13 x_14 x_15 x_16 x_17 x_18 x_19 x_20 x_21 x_22 x_23 x_24 x_25 x_26 x_27 x_28 x_29 x_30 x_31 x_32 x_33 x_34 x_35 x_36 x_37 x_38 x_39 x_40 x_41 x_42 x_43 x_44 x_45 x_46 x_47 x_48 x_49 x_50 x_51 x_52 x_53 x_54 x_55 x_56 x_57 x_58 x_59 x_60 x_61 x_62 x_63 x_64 ); * call graph(bigx :plottype meshstepc :heading 'Cancer Data' :rotation 90. :grid :d3axis :d3border /; :file 'rawdata.wmf' ); * call graph(bigx :plottype meshc :heading 'Cancer Data' /; :file 'rawdata.wmf' :rotation 90. :grid :d3axis :d3border ); bigx=transpose(bigx); docase1=1; docase2=1; if(docase1.ne.0)then; i=8; call cluster(bigx :print :k_mean i ); * call print(%iclus); ii=ranker(dfloat(%iclus)); newbigx=bigx(ii,); newtt2=tt(ii,); call compress; call graph(newbigx :plottype meshstepc :heading 'Clustered Data after transpose' :rotation 90. :grid :d3axis :d3border /; :file 'sorted_data.wmf' ); newlist=c8array(64:); do j=1,64; call pcopy(8,pointer(newtt2,j),64,pointer(newlist,j),1,-1); enddo; class=%iclus(ii); call print(' ':); call print('sum wss ',sum(%wss) :); call print('sum ave_wss ',sum(%ave_wss):); call tabulate(ii,class,newlist :rjname); endif; if(docase2.ne.0)then; n1=2; n2=12; ntotal=n2-n1+1; nclass =array(ntotal:); sumclass=array(ntotal:); s_av_wss =array(ntotal:); icount=0; do i=2,12; icount=icount+1; nclass(icount)=dfloat(i); call cluster(bigx :print :k_mean i ); * call print(%iclus); ii=ranker(dfloat(%iclus)); newbigx=bigx(ii,); newtt2=tt(ii,); call compress; * call graph(newbigx :plottype meshstepc :heading 'Clustered Data after transpose' :rotation 90. :grid :d3axis :d3border /; :file 'sorted_data.wmf' ); newlist=c8array(64:); do j=1,64; call pcopy(8,pointer(newtt2,j),64,pointer(newlist,j),1,-1); enddo; class=%iclus(ii); call print(' ':); sumclass(icount)=sum(%wss); s_av_wss(icount) =sum(%ave_wss); call print('sum wss ',sumclass(icount) :); call print('sum wss ',s_av_wss(icount) :); call tabulate(ii,class,newlist :rjname ); enddo; call tabulate(nclass sumclass, s_av_wss :rjname); call graph(nclass sumclass s_av_wss :plottype xyplot :nolabel :heading 'Sum of within class sum of squares'); endif; b34srun; CMAXF1 Constrained maximization of function using zxmwd. The CMAXF1 function provides a quick way to maximize a constrained function using the Quasi-Newton Method. If the functional value is multiplied by -1.0, a minimum can be obtained. A simple setup for a maximum / minimum is: call cmaxf1(func :name test :parms x1 x2 :ivalue rvec :lower lvalues :upper uvalues :print); where func is a scalar computed with the user MATRIX program test and x1 and x2 are parameters. Initial guess values for x1 and x2 are in the real vector rvec. For example the minimum of func = -3.*x2**2. + 4*x1**2 - x2 + 2.*x1; with answers -.2500, .1667 and func = -.3333 where -1. LE x1 LE 0. and 0. LE x2 LE 1. can be found with the commands: b34sexec matrix; program test; func=(-1.0)*((-3.)*x2**2. + 4*x1**2 - x2 + 2.*x1); call outstring(3,3,'Function to be minimized'); call outdouble(36,3,func); return; end; rvec=array(2:-1.2 1.0); ll=array(2:-1.,0.0); uu=array(2:.0 ,1.0 ); call cmaxf1(func :name test :parms x1 x2 :lower ll :upper UU :ivalue rvec :print); b34srun; The function name (func) the program name (test) and the parms are required to be passed. If there is a concern that the function has more than one minimum, the NLSTART command can be used to investigate a larger number of starting values. This feature of CMAXF1 makes it quite valuable. For example: b34sexec matrix; program test; func=-3.*x2**2. + 4*x1**2 - x2 + 2.*x1; return; end; n=2; k=10; a=array(n:-2. 2.); b=array(n:.5 2.); ll=array(2:-1.,0.0); uu=array(2:.0,1.0 ); call nlstart(a,b,k,s); do i=1,k rvec=s(,i); call cmaxf1(func :name test :parms x1 x2 :lower ll :upper UU :ivalue rvec :print); enddo; b34srun; ********************************************************** Required: func - Function name :name pgmname - User program to determine func :parms v1 v2 - Parameters in the model. These parameters must be in the function in the user program pgmname that determines func. The keyword :parms MUST be supplied prior to all keywords except :name. :lower ll - Vector of lower values for parameters. :upper uu - Vector of upper values for parameters Optional keywords for CMAXF1 are: :print - Print results :ivalue rvec - Determines initial values. rvec must be a vector containing the number of elements equal to the number of parameters supplied. Default = .1. :nsig i - Sets number of digits of accuracy for convergence. Default = 4. :nstart n - Number of starting points. Default = min(2**n+5,100) where n = number of parameters. CMAXF1 automatically creates the following variables. %coef - a vector containing the parameters. %nparm - a vector with coefficient names. %nsig - estimate of # of significant values. %func - final value of function. CMAXF2 Constrained maximization of function using dbconf/g. The CMAXF2 function provides a way to maximize a constrained function using the Quasi-Newton Method. If the functional value is multiplied by -1.0, a minimum can be obtained. CMAX2 uses IMSL routines dbconf & dbcong. A simple setup for a maximum / minimum is: call cmaxf2(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); If the gradiant is known the call is call cmaxf2(func grad :name test test2 :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); where func is a scalar computed with the user MATRIX program test and x1 and x2 are parameters. Initial guess values for x1 and x2 are in the real vector rvec. For example the minimum of FUNC = 100.*(x2-x1*x1)**2. + (1.-x1)**2. can be found with the commands: b34sexec matrix; program test; func=-1.0*(100.*(x2-x1*x1)**2. +(1.-x1)**2.); return; end; rvec=array(2:-1.2 1.0); ll=array(2:-2.,-1.0); uu=array(2:.5,2.0 ); call cmaxf2(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); b34srun; The function name (func) the program name (test) and the parms are required to be passed. If there is a concern that the function has more than one minimum, the NLSTART command can be used to investigate a number of starting values. For example: b34sexec matrix; program test; func=-1.0*(100.*(x2-x1*x1)**2. + (1.-x1)**2.); return; end; n=2; k=10; a=array(n:-2. 2.); b=array(n:.5 2.); call nlstart(a,b,k,s); do i=1,k rvec=s(,i); call cmaxf2(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); enddo; b34srun; Note that in the default mode, the commands for cmaxf1 and cmaxf2 are the same. The cmaxf2 command can optionally pass the name of the gradiant array after the func name and the name of the gradiant subroutine after the function subroutine. The set up for this optional mode is: b34sexec matrix; program test; func=(-1.0)*(100.*(x2-x1*x1)**2. + (1.-x1)**2.); call outstring(3,3,'Function'); call outdouble(36,3,func); call outdouble(4, 4, x1); call outdouble(36,4, x2); return; end; program der; g(1)= (400.0*(x2-x1*x1)*x1) + (2.0*(1.0-x1)); g(2)= -200.0*(x2-x1*x1); return; end; call print(test,der); rvec=array(2:-1.2, 1.0); ll= array(2:-2. ,-1.0); uu= array(2:.5 , 2.0); call echooff; call cmaxf2(func g :name test der :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); b34srun; ******************************************************** Required: func - Function name. Optionally the gradiant variable name can be supplied. :name pgmname - User program to determine func and optionally the program to determine the gradiant. :parms v1 v2 - Parameters in the model. These parameters must be in the function in the user program pgmname that determines func. The keyword :parms MUST be supplied prior to all keywords except :name. :lower rvec - Vector of lower values for parameters. :upper rvec - Vector of upper values for parameters Optional keywords for CMAXF2 are: :print - Print results. :ivalue rvec - Determines initial values. rvec must be a vector containing the number of elements equal to the number of parameters supplied. Default = .1. :xscale vec - Vector of n elements to scale x. Default = 1.0 :fscale real - Functional scaling. Default = 1.0. :ngood int - Sets number of good digits in the function. :maxit int - Maximum number of iterations. Default = 100. :maxfun int - Maximum number of function evaluations. Default = 400 :maxg int - Maximum number of gradiant evaluations. Default = 400 :gradtol real - Scaled gradiant tolerance. Default = eps**(1/3). :steptol real - Scaled step tolerance. Default = eps**(2/3). :rftol real - Relative functional tolerance. Default = max(1.0d-20,eps**(2/3)). :aftol real - Absolute functional tolerance. Default = max(1.0d-20,eps**(2/3)). :fctol real - False convergence tolerance. Default = 100.*eps. :maxsteps real - Maximum allowable step size. Default = (1000*max(tol1,tol2)) where tol1=sqrt(sum (xscale(i)*ivalue(i))**2) for i=1,n tol2 = 2-norm of XSCALE :ihessian key - where key is 0 to initialize hessian to identity matrix. This is default. If key NE 0, hessian initialized to max(|f(XGUESS|,FSCALE)*XSCALE(i) Warning: If you are not sure how to change a parameter, use the default. CMAXF2 automatically creates the following variables %coef - a vector containing the parameters. %nparm - a vector with coefficient names %se - a vector containing parameter standard errors %t - a vector containing parameter t scores %hessian - hessian matrix %grad - estimate of gradiant at final parameter values %func - final value of function Comments: CMAXF2 uses the IMSL routines dbconf & dbcong which are based the Scittkowski routine NLPQL. Both routines use the quasi-Newton method. The solution is updated according to the BFGS approach. For further references see the IMSL documentation. CMAXF3 Constrained maximization of function using db2pol. The CMAXF3 function provides a way to maximize a function using function comparison. No smoothness is assumed. While this approach is not efficient for smooth problems, it can be useful when the function is not smooth or to get starting values. The CMAXF3 function provides a way to maximize a constrained function using the complex method. Although no SE's are given, this command is useful to obtain starting values. If the functional value is multiplied by -1.0, a minimum can be obtained. A simple setup for a maximum / minimum is: call cmaxf3(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); where func is a scalar computed with the user MATRIX program test and x1 and x2 are parameters. Initial guess values for x1 and x2 are in the real vector rvec. For example the minimum of FUNC = 100.*(x2-x1*x1)**2. + (1.-x1)**2. can be found with the commands: b34sexec matrix; program test; func=-1.0*(100.*(x2-x1*x1)**2. + (1.-x1)**2.); return; end; rvec=array(2:-1.2 1.0); ll=array(2:-2.,-1.0); uu=array(2:.5,2.0 ); call cmaxf3(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); b34srun; The function name (func), the program name (test), and the parms are required to be passed. If there is a concern that the function has more than one minimum, the NLSTART command can be used to investigate a number of starting values. For example: b34sexec matrix; program test; func=-1.0*(100.*(x2-x1*x1)**2. + (1.-x1)**2.); return; end; n=2; k=10; a=array(n:-2. 2.); b=array(n:.5 2.); call nlstart(a,b,k,s); do i=1,k rvec=s(,i); call cmaxf3(func :name test :parms x1 x2 :ivalue rvec :lower ll :upper uu :print); enddo; b34srun; Note that in the default mode, the commands for cmaxf1, cmaxf2 and cmaxf3 are the same. ***************************************************** Required: func - Function name. Optionally the gradiant variable name can be supplied. :name pgmname - User program to determine func and optionally the program to determine the gradiant. :parms v1 v2 - Parameters in the model. These parameters must be in the function in the user program pgmname that determines func. The keyword :parms MUST be supplied prior to all keywords except :name. :lower rvec - Vector of lower values for parameters. :upper rvec - Vector of upper values for parameters. Optional keywords for CMAXF3 are: :print - Print results. :ivalue rvec - Determines initial values. rvec must be a vector containing the number of elements equal to the number of parameters supplied. Default = .1. :ftol real - Relative functional tolerance. Default =max(1.0d-20,eps**(2/3)). :maxit int - Maximum number of iterations. Default = 100. CMAXF3 automatically creates the following variables %coef - a vector containing the parameters. %nparm - a vector with coefficient names %func - final value of function The iterations proceed until: 1. # of iteratiions is reached. 2. func(best)-func(worst) LE ftol*(1+dabs(f(best)) 3. sum(1,...,(n+1))(f(i)-(sum(f(j))/(n+1))**2 LE ftol Warning: If you are not sure how to change a parameter, use the default. COMPRESS Compress workspace. call compress; To compress workspace. If call names(all); is given before and after this command, space compression can be observed. This command is usually never needed unless there are substaintal calculations being made. The variant n=100; call compress(n); will compress every 100 calls. call compress(:off); turns off compression even if call compress; or call compress(n); are found. call compress(:on); will turn on compression. call compress(:info); will provide information of settings. This command is useful for software developers. Note: The command call compress; will be ignored if it is used in a program, function or subroutine that is called as part of a nonlinear estimation command such as NLLLSQ, CMAX2 etc. The reason for this restriction is to avoid the possibility of data movement that is not known to the calling command. If memory management is needed in this case, use the solvefree command. The compress command will also be ignored if it is called from a function or from a subroutine that has been called by a user function. Related command: call printall; Notes: In do loops if call compress is given it should be placed at the top of the loop. However calling compress is usually not a good idea unless there are space limits. The reason for placing call compress right after the do statement is that all temps will be placed at the end of the workspace limiting the movement required. All non temp variables will stay in place low down in the workspace. Compacting the variables will speed execution. Example: /$ Illustrates call compress inside a LOOP /$ /$ Job # 1 runs saving space /$ /$ Note difference in space use /$ b34sexec matrix; call echooff; subroutine doit(n); x=rn(matrix(n,n:)); c=inv(x); return; end; count=1.; top continue; call compress; call doit(100); count=count+1.0; if(count.le.100.)go to top; b34srun; /$ /$ Job # 2 has call compress turned off /$ b34sexec matrix; call echooff; subroutine doit(n); x=rn(matrix(n,n:)); c=inv(x); return; end; count=1.; top continue; /$ call compress; call doit(100); count=count+1.0; if(count.le.100.)go to top; b34srun; CONSTRAIN Subset data based on range of values. call constrain(x,y,z:var z :lower .1 :upper 10.); Returns x, y and z values where z is in the range .1 to 10. If upper is not supplied it defaults to 1.0d+32. If lower is not supplied it defaults to -1.0d+32. Meld and constrain can be used to look at planes of more than 2d objects. H. H. Stokes is in debt to Stan Cohen the developer of Speakeasy for the idea for meld and constrain. Meld in b34s works like the simular command in Speakeasy. Constrain in Speakeasy allows multiple input testing. The variable tested in constrain must be real*8. If :lower and :upper are missing :var checks z against missing. A variable not tested against can be real*8, char*8 or integer. Constrain does not work for real*16 data at this time. Test Problem: b34sexec matrix; i=array(:1. 2. 3.); j=array(:4.,5.,6.); k=array(:7.,8.,9.); call tabulate(i,j,k); call meld(i,j,k); f=i**2.+j**2.+k**2.; call tabulate(i,j,k,f); call constrain(i,j,k,f:var i :lower 2.); call tabulate(i,j,k,f); call constrain(i,j,k,f:var k :upper 8.); call tabulate(i,j,k,f); b34srun; CONTRACT Contract an array call contract(old,ibegin,iend) Will contact a character*1 array. old = Character*1 string. ibegin = Integer pointer to a substring iend = Integer pointer to end of a substring old will be changed to have elements ibegin-iend removed. Example: b34sexec matrix; call character(cc,'This is a test'); call print(cc); call ilocatestr(cc,'is',istart,iend); call contract(cc,newcc,istart,iend); call print(newcc); b34srun; b34sexec matrix; call character(cc,'This is a test'); call print(cc); call ilocatestr(cc,istart,iend); i=integers(istart,iend); subs=cc(i); call print(subs); call contract(cc,istart,iend); oldnewcc=cc; call print(cc); call character(new,'aaaissaa'); call expand(cc,new,1,8); call print(oldnewcc,cc); b34srun; COPY Copy an object to another object call copy((2./4.),half); Works the same as an assingment but allows the target to be calculated real time. call copy(xx,xy); xx a character*1, character*8, real*4, real*8, real*16, VPA, complex*16, complex*32 or integer*4 or integer*8 variable. xy target Note: The statement call copy(x,y(i)); will not work as intended!!! Here y(i) is a temp variable. An optional third argument allows a copy to be made to a level call copy(x,y,1); The above call copies x from the current level to level 1. call copy(x,y,-1); copies from level 1 to teh current level. Warning. A copy made to an upper level will be lost due to memory cleaning. The intention of this command is to stay away from the level but "hide" data below the current base level of 100. This option is for the expert user. Example: b34sexec matrix; x=2.; call copy(x,y); call print(y); vpax=vpa(rn(array(5:))); call copy(vpax,vpay); call print(vpax,vpay); i=integers(6); i8=i4toi8(i); call copy(i8,i8copy); call print(i,i8,i8copy); /; /; Copy to a level /; x=100.; call copy(x,y,1); call names(all); b34srun; Example showing a copy on the fly b34sexec matrix; /; shows passing a name to a routine at execution; /; User wants the name my_x_dat & my_Y_dat for the /; random walk series!! /; These sure look like economic series n=10000; data1= cusum(rn(array(n:))); data2= cusum(rn(array(n:))); subroutine test(data1,data2,name1,name2); call copy(data1,argument(name1)); call copy(data2,argument(name2)); call graph(argument(name1),argument(name2) :Heading 'This Model is Spurious!!' :nokey); call describe(argument(name1),argument(name2)); call olsq(argument(name1),argument(name2) :print); return; end; name1='my_y_dat'; name2='my_x_dat'; call test(data1,data2,name1,name2); b34srun; COPYLOG Copy file to log file. call copylog('file') Copies a file to log unit. By use of call system, call copyout and call copylog, external programs such as RATS can be called inside a MATRIX command do loop to further process data. COPYOUT Copy file to output file. call copyout('file') Copies a file to output unit. By use of call system, call copyout and call copylog external programs such as RATS can be called inside a MATRIX command do loop to further process data. COPYTIME - Copy time info from series 1 to series 2 call copytime(series1,series2); call copytime(series1,series2,offset); Copies time series info from series1 to series2. If series1 is not a time series, will set series2 to not be a time series. If only two arguments are provided, then series1 => base seties series2 => series that will have time info updated offset => How many periods ahead. offset must be ge 0 call copytime(series,series2); is equivalent to call copytime(series1,series2,0); Example: b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call describe(m1dp :print); xx=m1dp*2.; call describe(xx :print); call copytime(m1dp,xx); call describe(xx :print); call copytime(m1dp,xx,10); call describe(xx :print); b34srun; COPYF Copy a file from one unit to another call copyf(in,iout); Copies a file from, unit in to unit iout. Units in and iout must have been allocated. Units are not closed. Example: call copyf(4,77); Application calling Matlab /$ Running Matlab script under B34S Matrix /$ Datacards allows saving of a Matlab script. b34sexex options; b34srun; b34sexec matrix; datacards; x=rand(6) xi=inv(x); x*xi yy=[1 2 3 2 1] plot(yy) pause quit b34sreturn; call open(77,'test.m'); call rewind(77); call copyf(4,77); call system( 'start /w matlab /r test /logfile test.out':); call copyout('test.out'); b34srun; Discussion: The cards after "datacards;" and before "b34sreturn;" are matlab commands that are copied from unit 4, the default parmcards unit, to unit 77. CSPECTRAL Do cross spectral analysis. call cspectral( ); Does spectral analysis on two series. The command: call cspectral(x,y,sinx,siny,cosx,cosy, px,py,sx,sy,rp,ip,cs,qs,a,k,ph,freq :weights); calculates: sinx - sine transform for x siny - sine transform for y cosx - cosine transform for x cosy - cosine transform for y px - Periodogram for x py - Periodogram for y sx - Spectrum for x sy - Spectrum for y rp - real part of cross periodogram. ip - imag. part of cross periodogram. cs - cross spectrum qs - quadra spectrum a - amplitude k - coherency squared ph - phase freq - frequency. period = (2.0*pi())/afam(freq); The CSPECTRAL command has 18 arguments or 19 arguments depending on whether weights are supplied. For one series, see spectral command. The FFTPACK subroutines ezffti and ezfftf are used to perfrom a FFT and calculate the sine and cosin transfroms from which all other values are calculated. CSUB Call Subroutine call csub('NAME', arguments :options) Options :lengthargs intarray( ) :list lists all supported routines Comment: The routine is for the expert user with access to subroutine argument lists. A branch will be made to b34smatcsubc in sourc16.f. This routine can someday be used for a DLL branch. This feature is not implemented at this time. The design of this routine may change substantially in the future. COINT2 Cointegration Tests of Two Series call coint2( ); Does cointegration tests on two series using Engle-Granger method. See CATS for Johansen method.. call coint2(x,y,xname,yname,dfx,dfy,adfx,adfy, lagx,lagy,speedx,speedy,tspeedx,tspeedy, dfx2,dfy2,adfx2,adfy2,dflag, resid0,resid1,resid2,iprint); Tests for Cointegration using Engle Procedure and two series. COINT2 is a subroutine and must be loaded with call load(coint2); Arguments x = first series y = second series xname = name of first series set with call character(xname,' ') yname = name of second series set with call character(yname,' ') dfx = Unit root test for x dfy = Unit root test for y adfx = Augmented DF test for x for lag=dflag adfy = Augmented DF test for y for lag=dflag lagx = Number of lags of x lagy = Number of lags of y speedx = Speed of adjustment of x speedy = Speed of adjustment of y tspeedx= t stat of speedx tspeedy= t stat of speedy dfx2 = Unit root test for x RES dfy2 = Unit root test for y RES adfx2 = Augmented DF test for x RES for lag=dflag adfy2 = Augmented DF test for y RES for lag=dflag dflag = Lag of DF test resid0 = Residual for Cointegrating Eq resid1 = Residual for Equation 1 resid2 = Residual for Equation 2 iprint = 0 no print, = 1 print For a discussion of the analysis see Enders (1995,365-373). Test Case: COINT2 Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint2); /$ call print(coint2); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call echooff; lagx=1; lagy=1; dflag=4; call coint2(y,z,xname,yname,dfx,dfy, adfx,adfy,lagx,lagy,speedx,speedy,tspeedx,tspeedy, dfx2,dfy2,adfx2,adfy2,dflag,resid0,resid1,resid2,1); call print(speedx,speedy,tspeedx,tspeedy); b34srun; COINT2LM Cointegration Tests of Two Series, OLS, L1, MM call coint2LM(x,y,xname,yname,dfx,dfy,adfx,adfy, lagx,lagy,speedx,speedy,tspeedx,tspeedy, l1speedx,l1speedy,mmspeedx,mmspeedy dfx2,dfy2,adfx2,adfy2,dflag, resid0,resid1,resid2,iprint); Tests for Cointegration using Engle Procedure and two series. Gives OLS, L1 and Minimax. For Johansen method, see CATS routine. COINT2LM is a subroutine and must be loaded with call load(coint2lm); x = first series y = second series xname = name of first series set with call character(xname,' ') yname = name of second series set with call character(yname,' ') dfx = Unit root test for x dfy = Unit root test for y adfx = Augmented DF test for x for lag=dflag adfy = Augmented DF test for y for lag=dflag lagx = Number of lags of x lagy = Number of lags of y speedx = Speed of adjustment of x speedy = Speed of adjustment of y tspeedx= t stat of speedx tspeedy= t stat of speedy L1speedx= Speed of Adjustment of x L1 estimator L1speedy= Speed of Adjustment of y L1 estimator mmspeedx= Speed of Adjustment of x Minimax estimator mmspeedx= Speed of Adjustment of y Minimax estimator dfx2 = Unit root test for x RES dfy2 = Unit root test for y RES adfx2 = Augmented DF test for x RES for lag=dflag adfy2 = Augmented DF test for y RES for lag=dflag dflag = Lag of DF test resid0 = Residual for Cointegrating Eq resid1 = Residual for Equation 1 resid2 = Residual for Equation 2 iprint = 0 no print, = 1 print Test case COINT2LM Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix cbuffer=100000; call loaddata; call load(coint2LM); call print(coint2LM); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call echooff; lagx=1; lagy=1; dflag=4; call coint2LM(y,z,xname,yname,dfx,dfy, adfx,adfy,lagx,lagy,speedx,speedy,tspeedx,tspeedy, l1speedx,l1speedy,mmspeedx,mmspeedy, dfx2,dfy2,adfx2,adfy2,dflag,resid0,resid1,resid2,1); call print(speedx,speedy, tspeedx, tspeedy, l1speedx,l1speedy,mmspeedx,mmspeedy); b34srun; COINT2M Moving Cointegration of Two Series call coint2m(x,y,xname,yname,number,lagx,lagy, speedx,speedy,tspeedx,tspeedy); Routine to drive coint2 using windows of data COINT2M is a subroutine and must be loaded with call load(coint2m); x = Input series # 1 y = Input series # 2 xname = Name of x series yname = Name of y series number = Number of observations in moving model lagx = Number of lags of X lagy = Number of lags of y speedx = Moving Error correction coefficient for x speedy = Moving Error correction coefficient for y tspeedx = t Stat of speedx tspeedy = t stat of speedy Test Case: COINT2M Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint2); call load(coint2m); call print(coint2,coint2m); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call echooff; number=60; lagx=1; lagy=1; call coint2m(y,z,xname,yname,number,lagx,lagy, speedx,speedy,tspeedx,tspeedy); call graph(speedx,tspeedx :heading 'Enders Y Series Moving Error Correction'); call graph(speedy,tspeedy :heading 'Enders Z Series Moving Error Correction'); call tabulate(speedx,speedy,tspeedx,tspeedy); b34srun; COINT2ME Moving Cointegration of Two Series - Extended Version call coint2me(x,y,xname,yname,number,lagx,lagy, speedx,speedy,dfx,dfy,adfx,adfy,dfres1,dfres2, adfres1,adfres2,dflag); Routine to drive coint2 using windows of data. COINT2ME is a subroutine and must be loaded with call load(coint2me); Uses expanded Arg list x = Input series # 1 y = Input series # 2 xname = Name of x series yname = Name of y series number = Number of observations in moving model lagx = Number of lags of X lagy = Number of lags of y speedx = Moving Error correction coefficient for x speedy = Moving Error correction coefficient for y tspeedx = t-Stat for speedx tspeedy = t-stat for speedy dfx = Dickey Fuller Test on Raw Data Series x dfy = Dickey Fuller Test on Raw Data Series y adfx = Augmented Dickey Fuller Test Raw Data Series x lag=dflag adfy = Augmented Dickey Fuller Test Raw Data Series y lag=dflag dfres1 = Dickey Fuller Test on RES1 Data Series dfres2 = Dickey Fuller Test on RES1 Data Series adfres1 = Augmented Dickey Fuller Test RES1 Data Series lag=dflag adfres2 = Augmented Dickey Fuller Test RES2 Data Series lag=dflag dflag = Lags for augmented DF test Test Case COINT2ME Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint2); call load(coint2me); call print(coint2,coint2me); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call echooff; number=60; lagx=1; lagy=1; dflag=4; /$ Shows simple call /$ call coint2m(y,z,xname,yname,number,lagx,lagy,speedx, /$ speedy,tspeedx,tspeedy); /$ call coint2me(y,z,xname,yname,number,lagx,lagy,speedx, speedy,tspeedx,tspeedy,dfx,dfy,adfx,adfy,dfres1, dfres2,adfres1,adfres2,dflag); call graph(speedx,tspeedx :heading 'Enders Y Series Moving Error Correction'); call graph(speedy,tspeedy :heading 'Enders Z Series Moving Error Correction'); call graph(dfx,dfy,speedx,speedy); call graph( speedx,speedy,tspeedx,tspeedy); call tabulate(speedx,speedy,tspeedx,tspeedy,dfx,dfy, dfres1,dfres2); call tabulate(speedx,speedy,tspeedx,tspeedy,adfx,adfy, adfres1,adfres2); b34srun; COINT2M2 Moving Cointegration Two Series OLS, L1 Minimax call coint2m2(x,y,xname,yname,number,lagx,lagy, speedx,speedy,tspeedx,tspeedy,l1speedx,l1speedy, mmspeedx,mmspeedy,dfx,dfy,adfx,adfy,dfres1,dfres2, adfres1,adfres2,dflag); Routine to drive coint2 using windows of data. Uses expanded Arg list. OLS, L1 and Minimax Estimates COINT2M2 is a subroutine and must be loaded with call load(coint2m2); x = Input series # 1 y = Input series # 2 xname = Name of x series yname = Name of y series number = Number of observations in moving model lagx = Number of lags of X lagy = Number of lags of y speedx = Moving Error correction coefficient for x speedy = Moving Error correction coefficient for y tspeedx = t-Stat for speedx tspeedy = t-stat for speedy l1speedx= Moving Error correction L1 coefficient for x l1speedy= Moving Error correction L1 coefficient for y mmspeedx= Moving Error correction Minimax coefficient for x mmspeedy= Moving Error correction Minimax coefficient for y dfx = Dickey Fuller Test on Raw Data Series x dfy = Dickey Fuller Test on Raw Data Series y adfx = Augmented Dickey Fuller Test Raw Data Series x lag=dflag adfy = Augmented Dickey Fuller Test Raw Data Series y lag=dflag dfres1 = Dickey Fuller Test on RES1 Data Series dfres2 = Dickey Fuller Test on RES1 Data Series adfres1 = Augmented Dickey Fuller Test RES1 Data Series lag=dflag adfres2 = Augmented Dickey Fuller Test RES2 Data Series lag=dflag dflag = Lags for augmented DF test test Case COINT2M2 Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix cbuffer=100000; call loaddata; call load(coint2lm); call load(coint2m2); call print(coint2lm,coint2m2); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call echooff; number=60; lagx=1; lagy=1; dflag=4; /$ Shows simple call /$ call coint2m(y,z,xname,yname,number,lagx,lagy,speedx, /$ speedy,tspeedx,tspeedy); /$ call coint2m2(y,z,xname,yname,number,lagx,lagy,speedx, speedy,tspeedx,tspeedy,l1speedx,l1speedy, mmspeedx,mmspeedy,dfx,dfy,adfx,adfy,dfres1, dfres2,adfres1,adfres2,dflag); call graph(speedx,tspeedx :nokey :heading 'Enders Y Series Moving Error Correction'); call graph(speedy,tspeedy :nokey :heading 'Enders Z Series Moving Error Correction'); call graph(speedx,l1speedx,mmspeedx :nokey :heading 'Enders Z Series Moving Error Correction'); call graph(speedy,l1speedy,mmspeedy :nokey :heading 'Enders Z Series Moving Error Correction'); call graph(dfx,dfy,speedx,speedy :nokey); call graph( speedx,speedy,tspeedx,tspeedy :nokey); call tabulate(speedx,speedy,tspeedx,tspeedy,dfx,dfy, dfres1,dfres2); call tabulate(speedx,speedy,tspeedx,tspeedy,adfx,adfy, adfres1,adfres2); call tabulate(speedx,l1speedx,mmspeedx,speedy, l1speedy,mmspeedy); b34srun; COINT3 Moving Cointegration of Three Series call coint3(x,y,z,xname,yname,zname,dfx,dfy,dfz, adfx,adfy,adfz,lagx,lagy,lagz,speedx,speedy, speedz,tspeedx,tspeedy,tspeedz,dfx2,dfy2,dfz2, adfx2,adfy2,adfz2,dflag,resid0,resid1,resid2, resid3,iprint); Tests for Cointegration using Engle Procedure and three series COINT3 is a subroutine and must be loaded with call load(coint3); x = first series y = second series z = third series xname = name of first series set with call character(xname,' ') yname = name of second series set with call character(yname,' ') zname = name of third series set with call character(zname,' ') dfx = Unit root test for x dfy = Unit root test for y dfz = Unit root test for z adfx = Augmented DF test for x lag=dflag adfy = Augmented DF test for y lag=dflag adfz = Augmented DF test for z lag=dflag lagx = Number of lags of x lagy = Number of lags of y lagz = Number of lags of z speedx = Speed of adjustment of x speedy = Speed of adjustment of y speedz = Speed of adjustment of z tspeedx = t of Speed of adjustment of x tspeedy = t of Speed of adjustment of y tspeedz = t of Speed of adjustment of z dfx2 = Unit root test for x RES dfy2 = Unit root test for y RES dfy2 = Unit root test for y RES adfx2 = Augmented DF test for x RES lag=dflag adfy2 = Augmented DF test for y RES lag=dflag adfz2 = Augmented DF test for z RES lag=dflag dflag = Sets lag on DF test resid0 = Residual for Cointegrating Eq resid1 = Residual for Equation 1 resid2 = Residual for Equation 2 resid3 = Residual for Equation 3 iprint = 0 no print, = 1 print Test Case: COINT3 Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint3); call print(coint3); call character(xname,'Enders w Series'); call character(yname,'Enders y Series'); call character(zname,'Enders z Series'); call echooff; lagx=1; lagy=1; lagz=1; dflag=4; call coint3(w,y,z,xname,yname,zname,dfx,dfy,dfz, adfx,adfy,adfz,lagx,lagy,lagz,speedx,speedy,speedz, tspeedx,tspeedy,tspeedz,dfx2,dfy2,dfz2,adfx2,adfy2, adfz2,dflag,resid0,resid1,resid2,resid3,1); call print(speedx,speedy,speedz); call print(tspeedx,tspeedy,tspeedz); b34srun; COINT_SW Stock-Watson DOLS Analysis call coint_sw(xvar,maxlag,res1,ec_coef,ec_vec,const, shock,iprint); /; /; Estimate Stock-Watson (1993) Cointegration test /; See Stock, James and Mark Watson. /; "A Simple Estimator of Cointegrating Vectoprs in higher /; order systems" Econometrica (61(4) (1993) : pp 783-820 /; /; Setup for VECM (Vector Error Component Analysis) /; /; xvar = matrix of variables for VECM /; maxlag = Maximum lag is plus and minus direction /; res1 = Residual from DOLS equation. Check acf to /; see if maxlag is set high enough /; ec_coef = Error Correction Coefficients /; ec_vec = Error Correction Vector /; const = mean(y-ec_vec); /; shock = y-ec_vec-const /; iprint = ne 0 => print estimated equation /; /; Built October 2009 by Houston H. Stokes /; Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint_sw); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call character(zname,'Enders w Series'); call echooff; iprint=0; maxlag=4; call coint_sw(catcol(y,z,w),10,res1,ec_coef,ec_vec, const,shock,iprint); if(iprint.eq.0)then; call print('Stock-Watson DOLS Estimation with maxlag = ', maxlag:); call print('Estimate of Cointegrating Vector ',ec_coef); call print('Estimate of constant ', const:); call graph(ec_vec,y,shock :nolabel :grid :pgborder :nocontact :heading 'Error Correction Vector'); call graph(res1 :heading 'Residual from DOLS Model ' :grid :pgborder :nocontact); b34srun; COINT3ME Moving Cointegration of Three Series call coint3me(x,y,z,xname,yname,zname,number, lagx,lagy,lagz,speedx,speedy,speedz, tspeedx,tspeedy,tspeedz,dfx,dfy,dfz,adfx, adfy,adfz,dfres1,dfres2,dfres3,adfres1, adfres2,adfres3,dflag); Routine to drive coint2 using windows of data. Uses expanded Arg list. COINT3ME is a subroutine and must be loaded with call load(coint3me); x = Input series # 1 y = Input series # 2 z = Input series # 3 xname = Name of x series yname = Name of y series zname = Name of z series number = Number of observations in moving model lagx = Number of lags of X lagy = Number of lags of y speedx = Moving Error correction coefficient for x speedy = Moving Error correction coefficient for y speedz = Moving Error correction coefficient for z tspeedx = t-Stat for speedx tspeedy = t-stat for speedy tspeedz = t-stat for speedz dfx = Dickey Fuller Test on Raw Data Series x dfy = Dickey Fuller Test on Raw Data Series y dfz = Dickey Fuller Test on Raw Data Series z adfx = Augmented Dickey Fuller Test Raw Data Series x lag=dflag adfy = Augmented Dickey Fuller Test Raw Data Series y lag=dflag dfres1 = Dickey Fuller Test on RES1 Data Series dfres2 = Dickey Fuller Test on RES2 Data Series dfres3 = Dickey Fuller Test on RES3 Data Series adfres1 = Augmented Dickey Fuller Test RES1 Data Series lag=dflag adfres2 = Augmented Dickey Fuller Test RES2 Data Series lag=dflag adfres3 = Augmented Dickey Fuller Test RES3 Data Series lag=dflag dflag = Lags for augmented DF test Test case: COINT3ME Example: b34sexec options ginclude('b34sdata.mac') macro(coint6); b34srun; b34sexec matrix; call loaddata; call load(coint3); call load(coint3me); call print(coint3,coint3me); call character(xname,'Enders y Series'); call character(yname,'Enders z Series'); call character(zname,'Enders w Series'); call echooff; number=60; lagx=1; lagy=1; lagz=1; dflag=4; call coint3me(y,z,w,xname,yname,zname,number, lagx,lagy,lagz,speedx,speedy,speedz, tspeedx,tspeedy,tspeedz, dfx,dfy,dfz,adfx,adfy,adfz, dfres1,dfres2,dfres3, adfres1,adfres2,adfres3,dflag); call graph(speedx,tspeedx :heading 'Enders Y Series Moving Error Correction'); call graph(speedy,tspeedy :heading 'Enders Z Series Moving Error Correction'); call graph(speedz,tspeedz :heading 'Enders w series Moving Error Correction'); call tabulate(speedx,speedy,speedz, tspeedx,tspeedy,tspeedz); b34srun; CONTRIB MARSPLINE Leverage plots/analysis call contrib( ) Easy to use MARS Contribution Analysis. For a more advanced setup see help file for contrib2. subroutine contrib(iopt,message,medians,iversion,isave, ihp,iols,olscoef,igrid,ishow,fsv_info); /; /; Produce Contribution Charts for all explanatory variables in /; a saved Multivariate Adaptive Regression Splines MARS model. /; Operates on medians and range of focus variable. 300 points /; are generated. /; /; In place of medians, user can supply values /; /; Added arguments by HH Stokes 22 February 2009 /; /; Original contrib subroutine was developed by William Lattyak /; and is in wbsuppl. This routine has different arguments. /; /; ************************************************************ /; /; Arguments: /; /; iopt => =0 => use medians /; =1 => use user supplied message 'medians.' /; 123456789012 /; message => used in place of ', o=Medians]' if /; iopt=1 /; /; allowed lengths iversion = 1 22 /; iversion = 2 16 /; iversion = 3 20 /; iversion = 4 20 /; iversion = 5 11 /; medians=> the stored medians of the righthand side /; variables /; /; iversion settings /; 1 => leverage effect of target variable on YHat holding /; all others constant /; 2 => contribution effect of target variable unit increase /; on YHat->YHat(t)-YHat(1) /; 3 => additive contribution of target variable removing /; all others /; 4 => contribution knot effect of target var on /; YHat diff1(Yhat) /; 5 => cumulative contribution of target variable unit /; increase on YHat->YHat(t)-YHat(1) /; /; Note: isave saves info for each chart cchart ..... /; isave=0 => no saves /; isave=1 => sca fsv /; isave=2 => rats por /; isave=3 => both sca fsv and rats por /; ihp =0 => graphs are *.wmf /; ihp =1 => graphs are *.hp1 /; iols =0 => do not place OLS vector on graphs /; iols =1 => place ols vector on graphs /; Note: iols only works with iversion=1 /; olscoef => vector of OLS coefficients. /; if one more than # MARS variables => constant /; igrid => 0 => no grid, =1 => grid lines /; ishow => 0 => no show, =1 => show /; fsv_info => Comment to place in fsv file. Max of 64 /; characters. /; /; Note: /; names of fsv and por files are the same as the corresponding /; graph files /; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Example File: /; /; Generation of contrib charts automatically /; /; Job can be easily modified /; b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call echooff; call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); /; call print(contrib); m=6; _knots=20; _mi=2; _df=2.0; /; set left hand side call character(l_hand_s,'gasout'); /; Set right hand side call character(_args,'gasout{1 to m} gasin{1 to m}'); /; Few settings needed after this line except for message /; if iopt=1 call olsq(argument(l_hand_s) argument(_args):diag :print); olscoef=%coef; /; call marspline(argument(l_hand_s) argument(_args) :mathform :print :nk _knots :mi _mi :df _df :savemodel :xx :savex ); /; Analysis by observation of variables. call marsdiag(%xx,c_sums,r_sums,2,2,'test1.wmf'); call marsinfo; /$ *****************************************************$/ /$ Create contribution charts for righthand-side variables /$ *****************************************************$/ /; /; First save the data and get medians/means etc /; call lagmatrix( argument(_args) :noint :matrix tmat); _medians=array(nocols(tmat):); _means=_medians; do i=1,norows(_medians); call describe(tmat(,i)); _medians(i) =%median; _means(i) =%mean; enddo; iopt=1; iols=1; ihp=0 ; isave=3; iversion=1; igrid=1; ishow=1; fsv_info='Basic GASOUT MARS Model Estimated'; /; 1234567890123456789012 message=', vector values used ]'; call contrib(iopt,message,_medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info); b34srun; CONTRIBD Easy to use driver for contrib2 Switches that are set prior to call to contribd. See listing of contribs/contribd for default settings iopt => =0 => use medians =1 => use user supplied message 'medians.' 123456789012 message => used in place of ', o=Medians]' if iopt=1 medians=> the stored medians of the righthand side variables iversion settings 1 => leverage effect of target variable on YHat holding all others constant 2 => contribution effect of target variable unit increase on YHat->YHat(t)-YHat(1) 3 => additive contribution of target variable removing all others 4 => contribution knot effect of target var on YHat diff1(Yhat) 5 => cumulative contribution of target variable unit increase on YHat->YHat(t)-YHat(1) Allowed lengths iversion = 1 22 iversion = 2 16 iversion = 3 20 iversion = 4 20 iversion = 5 11 Note: Saves in cchart..... isave=0 => no saves isave=1 => sca fsv isave=2 => rats por isave=3 => both sca fsv and rats por ihp =0 => graphs are *.wmf ihp =1 => graphs are *.hp1 iols =0 => do not place OLS vector on graphs iols = 1 => place ols vector on graphs Note: iols only works with iversion=1 iols = 2 => Place ols and GAM vectors on graphs Note: iols only works with iversion=1 = -2 => suppress mars on graph. Mars material must be supplied iols = 3 => Place ols, MARS GAM and ppreg vectors on graphs Note: iols ne 0 only works with iversion=1 olscoef => vector of OLS coefficients. if one more than # MARS variables => constant igrid => 0 => no grid, =1 => grid lines ishow => 0 => no show, =1 => show fsv_info => Comment to place in fsv file. Max of 64 characters. %spline => from gamfit if iabs(iols) ge 2 %xgam => from gamfit if iabs(iols) ge 2 %gamcoef => from gamfit if iabs(iols) ge 2 %link => from gamfit if iabs(iols) ge 2 %vartype => from gamfit if iabs(iols) ge 2 %df => from gamfit if iabs(iols) ge 2 degmod => Degree of polynomial fit for gam forecasts. %ppmod => from ppreg if iols eq 3 Listing of program contribd program contribd; /; mars - ols - gam - ppreg /; /; Settings outside call to dcontrib /; /; MARS /; _knots=16; /; _mi=1; /; _ms=0; /; /; gam /; /; degmod=6; /; /; ppreg /; /; _m =4; /; iopt=0; /; iols=3; /; ihp=0; /; isave=3; /; iversion=1; /; igrid=1; /; ishow=1; /; /; specific settings /; /; call character(fsv_info,'M1 =Fish Model'); /; call character(l_hand_s,'m1'); /; call character(_args, /; 'shift m1{1 to 2} fw ssam1'); /; call character(_argsg, /; 'shift[factor,1] m1[predictor,3]{1 to 2} fw ssam1 '); /; call dcontrib; /; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if(do_tests.ne.0)then; call olsq(argument(l_hand_s) argument(_args) :diag :qr :print); olscoef=%coef; call marspline(argument(l_hand_s) argument(_args) :mathform :print :nk _knots :mi _mi :df _df :savemodel :xx :savex ); /; Analysis by observation of variables. call marsdiag(%xx,c_sums,r_sums,2,2,'test1.wmf'); call marsinfo; /$ **********************************************************$/ /$ Create contribution charts for righthand-side variables /$ **********************************************************$/ /; /; First save the data and get medians/means etc /; call lagmatrix( argument(_args) :noint :matrix tmat); _medians=array(nocols(tmat):); _means=_medians; do i=1,norows(_medians); call describe(tmat(,i)); _medians(i) =%median; _means(i) =%mean; enddo; call gamfit(argument(l_hand_s) argument(_argsg) :dist gauss :print :savex :punch_sur); %gamcoef=%coef; %xgam=%x; call ppreg(argument(l_hand_s) argument(_args) :savemodel :modname %test :print :m _m ); ppr_yhat=%yhat; endif; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if(do_ppexp.ne.0)then; if(do_ppexp.eq.2) call ppexp(argument(l_hand_s) argument(_args) :mm _mm :jj _jj :fei _fei :nei _nei :trm _trm :print ); if(do_ppexp.eq.1) call ppexp(argument(l_hand_s) argument(_args) :mm _mm :jj _jj :fei _fei :nei _nei :trm _trm ); call print('Exploratory Projection Pursuit Index',%ppindex); call ppexp_p(%xpa,%mm,%nob,0,'t2',%ppindex); /; call ppexp_p(%xpa,%mm,%nob,1,'t2',%ppindex); endif; if(do_tests.ne.0) call contrib2(iopt,message,_medians,iversion,isave,ihp,iols, olscoef,igrid,ishow,fsv_info, %spline,%xgam,%gamcoef,%link, %vartype,%df,degmod,%test); return; end; For further detail see help file for contrib2 CONTRIBI Load Added Files to do contrib analysis Example of use: call load(contrib); call contribi; Note: Contribi listed below program contribi; /; /; load routines to run contrib analysis /; call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); call load(polyfit); call load(polyval); call load(gamfore); call load(ppexp_p); call contribs; return; end; Complete example of analysis: b34sexec matrix; call echooff; call loaddata; call load(contrib); call contribi; /; /; specific settings /; _mi=1; _m=10; do_ppexp=1; call character(fsv_info,'1. Misrala Model'); call character(l_hand_s,'y'); call character(_args, 'x'); call character(_argsg, 'x[predictor,3]'); call contribd; b34srun; CONTRIBS Initialize switches for contribd This routine is called by contribi but can be used many times in a job to re-initialize settings. Listing of program contribs program contribs; /; /; quick way to run OLS, MARS, GAM, ppexp and PPREG /; /; call before call to contribd; /; /; mars - ols - gam - ppreg /; /; /; MARS _knots=16; _mi=1; _ms=0; /; /; gam /; _df=2.0; degmod=6; /; /; ppreg /; /; # of trees _m =4; iopt=0; iols=3; ihp=0; isave=3; iversion=1; igrid=1; ishow=1; /; /; ppexp_p /; do_ppexp=0; do_tests=1; /; sets number of solutions _mm=5; /; sets order of legenre _jj=2; /; Threshold for data space dimensionality reduction _fei=.1e-4; /; Max dimension of search space _nei = 1; /; Trimming Threshold _trm=.0; return; end; Complete example of analysis: b34sexec matrix; call echooff; call loaddata; call load(contrib); call contribi; /; /; specific settings /; _mi=1; _m=10; do_ppexp=1; call character(fsv_info,'1. Misrala Model'); call character(l_hand_s,'y'); call character(_args, 'x'); call character(_argsg, 'x[predictor,3]'); call contribl; call contribd; b34srun; For further detail see help file for contrib2 CONTRIBL List switches in effect for contribd For further detail see help file for contrib2 Complete example of analysis: b34sexec options ginclude('stattest.mac') member(MISRA1A); b34srun; b34sexec matrix; call echooff; call loaddata; call load(contrib); call contribi; /; /; specific settings /; _mi=1; _m=10; do_ppexp=1; call character(fsv_info,'1. Misrala Model'); call character(l_hand_s,'y'); call character(_args, 'x'); call character(_argsg, 'x[predictor,3]'); call contribl; call contribd; b34srun; CONTRIB2 MARS-OLS-GAM-PPREG Leverage Plots call contrib2(iopt,message,medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline, %xgam,%gamcoef,%link,%vartype,%df,%degmod, %ppmod); MARS-OLS-GAM-PPREG Leverage Plots Note: contrib2 is loaded with contrib with command call load(contrib); The driving routine contribd and automatic setup routine contribs and listing of switches contribl and initialization contribi are also loaded subroutine contrib2(iopt,message,medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline, %xgam,%gamcoef,%link,%vartype,%df,%degmod, %ppmod); /; /; Produce Contribution Charts for all explanatory variables in /; a saved Multivariate Adaptive Regression Splines MARS model. /; allows also showing OLS, OLS GAM and OLS GAM PPREG forecasts /; Operates on medians and range of focus variable. 300 points /; are generated. /; /; In place of medians, user can supply values /; /; Added arguments by H. H. Stokes 10 March 2009 /; /; Original contrib subroutine was developed by William Lattyak /; and is in wbsuppl. This routine has different arguments. /; /; ************************************************************ /; /; Arguments: /; /; iopt => =0 => use medians /; =1 => use user supplied message 'medians.' /; 123456789012 /; message => used in place of ', o=Medians]' if /; iopt=1 /; /; allowed lengths iversion = 1 22 /; iversion = 2 16 /; iversion = 3 20 /; iversion = 4 20 /; iversion = 5 11 /; medians=> the stored medians of the righthand side /; variables /; /; iversion settings /; 1 => leverage effect of target variable on YHat holding /; all others constant /; 2 => contribution effect of target variable unit increase /; on YHat->YHat(t)-YHat(1) /; 3 => additive contribution of target variable removing /; all others /; 4 => contribution knot effect of target var on /; YHat diff1(Yhat) /; 5 => cumulative contribution of target variable unit /; increase on YHat->YHat(t)-YHat(1) /; /; Note: chaves in cchart...... /; isave=0 => no saves /; isave=1 => sca fsv /; isave=2 => rats por /; isave=3 => both sca fsv and rats por /; ihp =0 => graphs are *.wmf /; ihp =1 => graphs are *.hp1 /; iols =0 => do not place OLS vector on graphs /; iols =1 => place ols vector on graphs /; Note: iols only works with iversion=1 /; iols = 2 => Place ols and GAM vectors on graphs /; Note: iols only works with iversion=1 /; = -2 => suppress mars on graph. Mars material must be /; supplied /; iols =3 => Place ols, GAM and ppreg vectors on graphs /; Note: iols ne 0 only works with iversion=1 /; iols =4 => Place ols, GAM MARS RF ppreg vectors on graphs /; Note: iols ne 0 only works with iversion=1 /; olscoef => vector of OLS coefficients. /; if one more than # MARS variables => constant /; igrid => 0 => no grid, =1 => grid lines /; ishow => 0 => no show, =1 => show /; fsv_info => Comment to place in fsv file. Max of 64 /; characters. /; %spline => from gamfit if iabs(iols) ge 2 /; %xgam => from gamfit if iabs(iols) ge 2 /; %gamcoef => from gamfit if iabs(iols) ge 2 /; %link => from gamfit if iabs(iols) ge 2 /; %vartype => from gamfit if iabs(iols) ge 2 /; %df => from gamfit if iabs(iols) ge 2 /; degmod => Degree of polynomial fit for gam forecasts. /; %ppmod => from ppreg if iols eq 3 /; Note: /; names of fsv and por files are the same as the corresponding /; graph files /; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ /; subroutine contrib2(iopt,message,medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline,%xgam,%gamcoef,%link, %ppmod); /; /; Produce Contribution Charts for all explanatory variables in /; a saved Multivariate Adaptive Regression Splines MARS model. /; allows also showing OLS, OLS GAM and OLS GAM PPREG forecasts /; Operates on medians and range of focus variable. 300 points /; are generated. /; /; In place of medians, user can supply values /; /; Note: if the variable si not in the MARS model, the variable /; chart will not be produced /; /; Added arguments by H. H. Stokes 22 February 2009 /; /; Original contrib subroutine was developed by William Lattyak /; and is in wbsuppl. This routine has different arguments. /; /; ************************************************************ /; /; Arguments: /; /; iopt => =0 => use medians /; =1 => use user supplied message 'medians.' /; 123456789012 /; message => used in place of ', o=Medians]' if /; iopt=1 /; /; allowed lengths iversion = 1 22 /; iversion = 2 16 /; iversion = 3 20 /; iversion = 4 20 /; iversion = 5 11 /; medians=> the stored medians of the righthand side /; variables /; /; iversion settings /; 1 => leverage effect of target variable on YHat holding /; all others constant /; 2 => contribution effect of target variable unit increase /; on YHat->YHat(t)-YHat(1) /; 3 => additive contribution of target variable removing /; all others /; 4 => contribution knot effect of target var on /; YHat diff1(Yhat) /; 5 => cumulative contribution of target variable unit /; increase on YHat->YHat(t)-YHat(1) /; /; isave=0 => no saves /; isave=1 => sca fsv /; isave=2 => rats por /; isave=3 => both sca fsv and rats por /; ihp =0 => graphs are *.wmf /; ihp =1 => graphs are *.hp1 /; iols =0 => Place mars vector on graphs /; iols =1 => place ols vector on graphs in addition to mars /; Note: iols only works with iversion=1 /; iols =2 => Place ols and GAM vectors on graphs in addition /; to mars /; iols =-2 => Suppress MARS plot. MARS needs to be estimated /; however /; iols =3 => Place ols, GAM and ppreg vectors on graphs /; Note: iols ne 0 only works with iversion=1 /; Note: iols ne 0 only works with iversion=1 /; iols =4 => Place ols, GAM MARS RF ppreg vectors on graphs /; Note: iols ne 0 only works with iversion=1 /; olscoef => vector of OLS coefficients. /; if one more than # MARS variables => constant /; igrid => 0 => no grid, =1 => grid lines /; ishow => 0 => no show, =1 => show /; fsv_info => Comment to place in fsv file. Max of 64 /; characters. /; %spline => from gamfit needed if iabs(iols) ge.2 /; %xgam => from gamfit needed if iabs(iols) ge.2 /; %gamcoef => from gamfit needed if iabs(iols) ge.2 /; %link => from gamfit needed if iabs(iols) ge.2 /; %df => from gamfit needed if iabs(iols) ge.2 /; Warning if MARSPLINE called prior to /; call contrib2 but after call gamfit, need to /; refresh %df since marspline outputs %df also /; %degmod => sets degree of polynomial for gam forecasting. /; usuallay set = 6 /; %ppmod => from ppreg if iols eq 3 /; /; Note: /; names of fsv and por files are the same as the corresponding /; graph files /; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The program contribd is loaded and privides a quick way to estimate OLS, MARS, GAM and PPREG. An example of its use is shown below: b34sexec matrix; call loaddata; call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); call load(polyfit); call load(polyval); call load(gamfore); call echooff; /; Estimates mars - ols - gam - ppreg and shows leverage plots /; /; Settings outside call to dcontrib /; /; MARS /; _knots=16; _mi=1; _ms=0; /; /; gam /; _df=2.0; degmod=6; /; /; ppreg /; _m =4; iopt=0; iols=3; ihp=0; isave=3; iversion=1; igrid=1; ishow=1; /; /; specific settings /; call character(fsv_info,'M1 =Fish Model'); call character(l_hand_s,'m1'); call character(_args, 'shift m1{1 to 2} fw ssam1'); call character(_argsg, 'shift[factor,1] m1[predictor,3]{1 to 2} fw ssam1 '); call contribd; ++++++++ The routine contribs makes calling even contribd easier. The following 6 statements will initialize the switches for dcontrib listed earlier and run OLS, MARS, GAM and PPREG using as defaults the settings listed. If for example one wants to allow for a possibility of 2 interactions add the line _mi=2; after call contribs; A very simple setup is: call contribs; call character(fsv_info,'M1 =Fish Model'); call character(l_hand_s,'m1'); call character(_args, 'shift m1{1 to 2} fw ssam1'); call character(_argsg, 'shift[factor,1] m1[predictor,3]{1 to 2} fw ssam1 '); call contribd; ++++++++ Example File 1 - Manual call to contrib2: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call echooff; call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); call load(polyfit); call load(polyval); call load(gamfore); /; call print(contrib); m=6; _knots=20; _mi=2; _df=2.0; iopt=0; iols=3; ihp=0 ; isave=3; iversion=1; igrid=1; ishow=1; fsv_info='Basic GASOUT MARS Model Estimated'; /; 1234567890123456789012 message=', vector values used ]'; /; set left hand side call character(l_hand_s,'gasout'); /; Set right hand side call character(_args,'gasout{1 to m} gasin{1 to m}'); call character(_args2,'gasout[predictor,3]{1 to m} gasin[predictor,3]{1 to m}'); /; Few settings needed after this line except for message /; if iopt=1 call olsq(argument(l_hand_s) argument(_args):diag :print); olscoef=%coef; /; call marspline(argument(l_hand_s) argument(_args) :mathform :print :nk _knots :mi _mi :df _df :savemodel :xx :savex ); /; Analysis by observation of variables. call marsdiag(%xx,c_sums,r_sums,2,2,'test1.wmf'); call marsinfo; /$ *****************************************************$/ /$ Create contribution charts for righthand-side variables /$ *****************************************************$/ /; /; First save the data and get medians/means etc /; call lagmatrix( argument(_args) :noint :matrix tmat); _medians=array(nocols(tmat):); _means=_medians; do i=1,norows(_medians); call describe(tmat(,i)); _medians(i) =%median; _means(i) =%mean; enddo; call gamfit(argument(l_hand_s) argument(_args2) :dist gauss :print :savex :punch_sur); %gamcoef=%coef; %xgam=%x; call ppreg(argument(l_hand_s) argument(_args) :savemodel :modname %test :print :m 4 :holdout 10 ); ppr_yhat=%yhat; call contrib2(iopt,message,_medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline, %xgam,%gamcoef,%link,%vartype,%df,6,%test); b34srun; Example File 2: b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; /; /; Illustrates OLS MARS an GAM Leverage Plots /; b34sexec matrix; call loaddata; call echooff; call olsq(lpeptide age bdeficit :print); call gamfit(lpeptide age[predictor,3] bdeficit[predictor,3] :dist gauss :print :savex :punch_sur); call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); call load(polyfit); call load(polyval); call load(gamfore); m=6; _knots=20; _mi=2; _df=2.0; iopt=0; iols=2; ihp=0 ; isave=3; iversion=1; igrid=1; ishow=1; fsv_info='Basic bdeficit MARS Model Estimated'; /; 1234567890123456789012 /; Needed if iopt=1 message=', vector values used ]'; /; set left hand side call character(l_hand_s,'lpeptide'); /; Set right hand side /; _args for MARS and OLS /; args2 for GAM call character(_args, 'age bdeficit'); call character(_args2,'age[predictor,3] bdeficit[predictor,3]'); /; Few settings needed after this line except for message /; if iopt=1 call olsq(argument(l_hand_s) argument(_args):diag :print); olscoef=%coef; /; call marspline(argument(l_hand_s) argument(_args) :mathform :print :nk _knots :mi _mi :df _df :savemodel :xx :savex ); /; Analysis by observation of variables. call marsdiag(%xx,c_sums,r_sums,2,2,'test1.wmf'); call marsinfo; /$ *****************************************************$/ /$ Create contribution charts for righthand-side variables /$ *****************************************************$/ /; /; First save the data and get medians/means etc /; call lagmatrix( argument(_args) :noint :matrix tmat); _medians=array(nocols(tmat):); _means=_medians; do i=1,norows(_medians); call describe(tmat(,i)); _medians(i) =%median; _means(i) =%mean; enddo; call gamfit(argument(l_hand_s) argument(_args2) :dist gauss :print :savex :punch_sur); %gamcoef=%coef; %xgam=%x; call ppreg(argument(l_hand_s) argument(_args) :savemodel :modname %test :print :m 4 :holdout 10 ); ppr_yhat=%yhat; call contrib2(iopt,message,_medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline, %xgam,%gamcoef,%link,%vartype,%df,6,%test); b34srun; Example # 3: /; Study of SO4 and Latitude and Longitude /; Data Studied by Dong Xiang at SAS Institute /; 'Fitting Generalized Additive Models with GAM Proc' /; SAS paper 256-26 b34sexec options ginclude('b34sdata.mac') member(gam_6); b34srun; b34sexec matrix; call loaddata; call load(gamplot); /; /; needed for contrib plots /; call load(marsdiag :staging); call load(marsinfo :staging); call load(contrib); call load(polyfit); call load(polyval); call load(gamfore); call echooff; iopt=0; iols=3; ihp=0 ; isave=3; iversion=1; igrid=1; ishow=1; _df=2.; _mi=2; _knots=20; _m=50; fsv_info='SO4 and Latitude and Longitude'; /; 1234567890123456789012 message=', vector values used ]'; /; set left hand side call character(l_hand_s,'so4'); /; Set right hand side call character(_args,'latt long'); call olsq(argument(l_hand_s) argument(_args) :diag :print); olscoef=%coef; call marspline(argument(l_hand_s) argument(_args) :mathform :print :nk _knots :mi _mi :df _df :savemodel :xx :savex ); /; Analysis by observation of variables. call marsdiag(%xx,c_sums,r_sums,2,2,'test1.wmf'); call marsinfo; /$ *****************************************************$/ /$ Create contribution charts for righthand-side variables /$ *****************************************************$/ /; /; First save the data and get medians/means etc /; call lagmatrix( argument(_args) :noint :matrix tmat); _medians=array(nocols(tmat):); _means=_medians; do i=1,norows(_medians); call describe(tmat(,i)); _medians(i) =%median; _means(i) =%mean; enddo; call gamfit(argument(l_hand_s) argument(_args) :dist gauss :print :savex :punch_sur); %gamcoef=%coef; %xgam=%x; call ppreg(argument(l_hand_s) argument(_args) :savemodel :modname %test :print :m 4 :holdout 10 ); ppr_yhat=%yhat; call contrib2(iopt,message,_medians,iversion,isave,ihp, iols,olscoef,igrid,ishow,fsv_info,%spline, %xgam,%gamcoef,%link,%vartype,%df,6,%test); b34srun; Complete example of analysis using driving routines: b34sexec matrix; call echooff; call loaddata; call load(contrib); call contribi; /; /; specific settings /; _mi=1; _m=10; do_ppexp=1; call character(fsv_info,'1. Misrala Model'); call character(l_hand_s,'y'); call character(_args, 'x'); call character(_argsg, 'x[predictor,3]'); call contribd; b34srun; CPERIOD Normalized Cumulative Periodogram call cperiod(series,'sname',c_period,c_p_freq,idrop); / subroutine cperiod(x,name,c_period,c_p_freq,idrop); /; /; Normalized Cumulative Periodogram /; /; Box-Jenkins-Rensel (2008,347-350) suggests calculation /; of cumulative Periodogram to test detect periodic /; nonrandomness /; /; See Jenkins and Watts (1968, 235) for significance bands /; /; For significance of .95 and .75 lamda = 1.36 and 1.02 /; .99 and .90 lamda = 1.63 and 1.22 /; band is +- lamda/sqrt(n/2)-1)) /; /; Command built October 2009 by Houston H. Stokes /; /; x = series to test /; name = name of series /; c_period = normalized cumulative periodogram /; c_p_freq = frequency of c_period /; idrop = Number of c_period values to drop /; /; name of file is 'c_n_period.wmf' /; Example: b34sexec options ginclude('gas.b34'); b34srun; /$ /$ Job tests c_period Command /$ b34sexec matrix; call loaddata; call load(cperiod); idrop=0; call cperiod(gasout,'gasout',c_period,c_p_freq,idrop); x=rn(gasout); call cperiod(x, 'ran()',c_period,c_p_freq, idrop); b34srun$ CSV Read and Write a CSV file call csv(:readfile 'mycsv.csv' ); call csv(:readfile 'mycsv.csv' ); call csv(:readfile 'mycsv.csv' :var x); call csv(:readfile 'mycsv.csv' :var x); call csv(:writefile 'mycsv.csv' :var x1 x2 x3); call csv(:writefile 'mycsv.csv' :var x1 x2 x3 :nonames :nodatestamp); Reads and writes csv files. Real*8 or character*8 data may be supplied. A row of names usually must proceed the data. Comments can be supplied before the names row. Usage Notes: Missing data in csv files in many cases is non standard. ,., and ,-, are recoded as missing A number of users of Excel save numbers such as 1781 as 1,781. Such a number will be saved in the csv file as "1,781" and will be classified as character*8 by the b34s csv reader. The fix is involves reformatting the cells in that column to type number. The same goes for numbers saved as $1,781. Character*8 variables with blanks will be recoded with _. For example John S will be recoded as John_S so that it can be loaded back into the main part of b34s. The call csv command can also read R read.table files with header=T Note: The dataread command will read real*8, real*16 and character*8 data and allows optional format control. Options, some of which may be required. Either :readfile or :writefile must be supplied first. :readfile 'filename Supplies a file name of a csvfile and opens for indicated action. :writefile 'filename' Supplies a file name of a csvfile and opens for indicated action. :nodatestamp Will not put a datestamp comment in file if there is a :writefile. :nonames On a write does not place names in the file. This is useful with the :add option. If :nonames is present a blank line between writes is suppressed. Usually when :nonames is used :nodatestamp is also supplied. If :nonames is supplied for the first write, then the call dataread command must be used. :comment Supply a comment file of 1-80 charactacters. :var If a :writefile then :var is required to supply variable names. Up to 255 can be supplied. If a matrix is supplied, then only one variable name is allowed. :nsmissing Sets missing to N/S for real data for a write. :missing Sets missing to blank for a write for real data. :add Will add data at the end of the CSV file. The file cannot be read back into B34S at this time but can be read into Excel but not with 100% reliability since all data is seen as just cells. The :add option allows the matrix command to "dump" results into one file that can be read into Excel for further copying into Word etc. Note that if :add is supplied the file must exist and be of the csv type. Use :add with caution. If :nonames is supplied as well as :add, more data is loaded into the file but the names header is not supplied. :echo Can be set is there are problems reading the csv file. Will echo what is being convcerted. Problems such as blanks for character data have to be fixed in the Excee file. Variables created: %series created if :readfile is supplied Example of loading data into the matrix command and graphing. b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; people=c8array(:'houston','diana','will','bobby'); ii=dfloat(integers(10)); call csv(:writefile 'mycsv.csv' :comment 'This is a test' :var gasout gasin people ii); x=rn(matrix(10,5:)); call echooff; do i=1,nocols(x); call print(i,mean(x(,i))); enddo; /; call print(x); call csv(:writefile 'mycsv2.csv' :comment 'This is a test matrix' :var x ); call cleardat; call names; call csv(:readfile 'mycsv.csv'); call names; call print(%series); n= norows(%series); call tabulate(gasout,gasin,people,ii); call print(mean(gasout),mean(gasin)); do i=1,n; if(kind(eval(%series(i))).eq.8)then; g=goodrow(eval(%series(i))); call copy(g,argument(%series(i))); call print(' ':); call describe( eval(%series(i):)); call graph( eval(%series(i):)); endif; enddo; /; reading matrix call cleardat; call names; call csv(:readfile 'mycsv2.csv'); call names; /; Tests with alternative file saving of missing data x=rn(array(5:)); y=rn(array(10:)); call csv(:writefile 'mycsv3.csv' :var x y :nsmissing ); call csv(:writefile 'mycsv4.csv' :var x y :missing ); call cleardat; call csv(:readfile 'mycsv3.csv' ); call tabulate(X,Y); call cleardat; call csv(:readfile 'mycsv4.csv' ); call tabulate(X,Y); b34srun; Example of saving 200 series into Stata /; /; read in Stata with /; insheet double using c:\b34swork\mycsv2.cvs /; b34sexec matrix; x=rn(matrix(1000,200:)); call csv(:writefile 'mycsv2.csv' :var x :nodatestamp); b34srun; Examples of reading into a b34s data set. Max of 255 serries. b34sexec matrix; call csv(:readfile 'mycsv2.csv'); n=norows(%series); /; space between names c=c8array(n*2:); i=integers(1,n); j=integers(1,2*n,2); c(j)=%series(i); call makedata(argument(c) :file 'new.b34'); b34srun; b34sexec options include('new.b34'); b34srun; Alternate makedata. call makedata(%series :file 'new.b34' :add :member(tt)); Example of a *.csv file File built 20/ 4/05 at 19:50:26 by b34s,,,, Real*8 and character*8 data loaded,,,,, Missing data in last line,,,,, a,b,c,d,e,names 1,4,6,1,2,Houston 2,5,7,1,2,Bobby 3,6,8,1,2,Diana 4,7,9,,2,Will Example of reading a R read.table file with tab characters or , delimiters %B34SLET in = "book1.csv" $ %B34SLET out = "_myb34s.b34" $ %B34SLET IN ="'d:\therbook\cancer.txt'" $ %B34SLET OUT ="'_myb34s.b34'" $ b34sexec matrix; call csv(:readfile %b34seval(&in)); %_n=norows(%series); %_c=c8array(%_n*2:); %_i=integers(1,%_n); %_j=integers(1,2*%_n,2); %_c(%_j)=%series(%_i); /; If less that 98 series can made data step. /; Otherwise can read any number of series into /; matrix command. call makedata(argument(%_c) :file %b34seval(&out)); b34srun; b34sexec options include(%b34seval(&out)); b34srun; /; b34sexec list; b34srun; Adding to a CSV file. /; Writes with names and header but adds lines only b34sexec matrix; n=10; k=3; x=rn(array(n,k:)); call csv(:writefile 'test.csv' :var x); x=x*10.; call csv(:writefile 'test.csv' :var x :add :nonames :nodatestamp); b34srun; /; Writes without names b34sexec matrix; n=10; k=3; x=rn(array(n,k:)); call csv(:writefile 'test2.csv' :var x :nodatestamp :nonames ); x=x*10.; call csv(:writefile 'test2.csv' :var x :add :nonames :nodatestamp); b34srun; /; Test reads b34sexec matrix; call csv(:readfile 'test.csv'); call names(all); do i=1,norows(%series); call print(argument(%series(i))); enddo; b34srun; /; Reading a noname file using call read b34sexec matrix; n=20; k=3; call open(71,'test2.csv'); tdata=array(k,n:); call read(tdata,71); call close(71); tdata=transpose(tdata); call print(tdata); b34srun; CUMFCST - Calculate Cumulative Forecast call cumfcst(y, fcst, crfcst); y = actual data that may contain 0.0 values fcst = forecast of actual data crfcst = cumlulative forecast Example showing data and a test subroutine to show logic b34sexec data heading('cumfcst Test'); input obs y fcst _TCUMF ; label obs = 'Observation #'; label y = 'Actual Data '; label fcst = 'Forecast value'; label _tcumf = 'Test value '; datacards; 1 440 . . 2 2200 . . 3 2640 1980 . 4 2200 1980 1980 5 880 1980 1980 6 4400 1980 1980 7 440 1980 1980 8 2200 1980 1980 9 0 1980 . 10 440 1980 3960 11 1760 1709 1980 12 1320 1747 1709 13 2640 1780 1747 14 2640 1809 1780 15 1760 1834 1809 16 2640 1856 1834 17 2640 1875 1856 18 5280 1890 1875 19 1760 1904 1890 20 440 1916 1904 21 0 1926 . 22 0 1926 . 23 1320 1926 5767 24 3520 1476 1926 25 2200 1539 1476 26 880 1595 1539 27 3080 1646 1595 28 0 1691 . 29 880 1691 3337 30 1760 1520 1691 31 0 1578 . 32 4840 1578 3099 33 0 1442 . 34 440 1442 3021 35 0 1345 . 36 1760 1345 2787 37 440 1272 1345 38 2200 1349 1272 39 4400 1421 1349 40 0 1487 . 41 440 1487 2908 42 440 1378 1487 43 6600 1448 1378 44 880 1512 1448 45 0 1571 . 46 1760 1571 3083 47 440 1437 1571 48 0 1502 . 49 4840 1502 2940 50 440 1388 1502 51 2640 1458 1388 52 440 1521 1458 53 1760 1579 1521 54 1760 1632 1579 55 0 1679 . 56 0 1679 . 57 880 1679 4989 58 3960 1348 1679 59 2640 1420 1348 60 440 1487 1420 61 880 1548 1487 b34sreturn; b34srun; b34sexec matrix; call loaddata; call echooff; subroutine crfcst(y, fcst, %crfcst); /; /; Shows logic of "hardwired" cumfcst command /; /; y = actual data that may contain 0.0 values /; fcst = forecast of actual data /; %crfcst = cumlulative forecast /; /; Routine built by William Lattyak November 2010 /; /; +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ fcst=fcst(integers(1,norows(y))); _zsum=0.; _tfcst=recode(fcst,missing(),0.); %crfcst=_tfcst; %crfcst=%crfcst*0.; %crfcst=recode(%crfcst,0.,missing()); _ibegin=0; do %iiii=1,norows(fcst); if ((sfam(y(%iiii)) .ne. 0.0) .and. (sfam(_tfcst(%iiii)) .ne. 0.0) )then; if(_ibegin .eq. 0) then; _zsum=sfam(missing()); _ibegin=1; endif; %crfcst(%iiii)=_zsum; _zsum=0.; endif; _zsum=_zsum + _tfcst(%iiii); enddo; return; end; fcst2=fcst; call crfcst(y,fcst, test1); call cumfcst(y,fcst2,test2); call tabulate(y,fcst,test1,test2,_tcumf :title 'test1 = subroutine value. test2 = internal command'); b34srun; DATA_ACF - Calculate ACF and PACF Plots subroutine data_acf(x,heading1,nacf); /$ /$ Display series and ACF /$ /$ x = Series to display /$ heading = Heading for series /$ nacf = Number of ACF and PACF /$ /$ Note: Can be called alone or under dataview /$ /$ Use BJ_IDEN ACF and PACF Listing /$ *********************************************** /$ Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(data_acf); call character(cc,'ACF & PACF of GASOUT'); call data_acf(gasout,cc,60); b34srun; DATA2ACF - Calculate ACF and PACF Plots file argument subroutine data2acf(x,heading1,nacf,file); /$ /$ Display series and ACF /$ /$ x = Series to display /$ heading = Heading for series /$ nacf = Number of ACF and PACF /$ file = file name /$ /$ Note: Can be called alone or under dataview /$ /$ *********************************************** /$ Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(data_acf); call character(cc,'ACF & PACF of GASOUT'); call data2acf(gasout,cc,60,'gas_acf.wmf'); b34srun; DATAFREQ - Data Frequency Calculate Data Frequencies for various options call datafreq(x,table :key); Required: x - series (must be real*8). table - Frequency count :key - must be set either equal, equaluser, usercutoff, userclass, count :equal k midpts => k equal intervals whose midpoints are in midpts where data sets xlow and xhigh. :equaluser k midpts xlow xhigh => k equal intervals whose midpoints are in midpts but xlow and xhigh are used. :usercutoff cutpts => k counts in table where there are k-1 cut points supplied :userclass classmk clhw => Class marks are input in classmk and class half width in clhw. :count values => Will place in table the exact number of cases that match values. sum(table) is le number of obs. Example: b34sexec matrix; * IMSL test cases for one-way Frequency analysis; x=array(:0.77, 1.74, 0.81, 1.20, 1.95, 1.20, 0.47, 1.43, 3.37, 2.20, 3.00, 3.09, 1.51, 2.10, 0.52, 1.62, 1.31, 0.32, 0.59, 0.81, 2.81, 1.87, 1.18, 1.35, 4.75, 2.48, 0.96, 1.89, 0.90, 2.05); call datafreq(x,table1 :equal 10 midpts1); call tabulate(table1,midpts1); xlow=.5; xhigh=4.5; call datafreq(x,table2 :equaluser 10 midpts2 xlow xhigh); call names(all); call tabulate(table2,midpts2); cutpts=array(:.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5); call datafreq(x,table3 :usercutoff cutpts); call tabulate(table3,cutpts); classmk=array(:.25 .75 1.25 1.75 2.25 2.75 3.25 3.75 4.25 4.75); clhw=.25; call datafreq(x,table4 :userclass classmk clhw); call tabulate(table4,classmk); b34srun; DATAREAD Read a user flat file into matrix command directly Provide a facility to read one dimensional data into matrix command in a flexible manner that does not require the more general read command. The dataread command allows user to easily "break" 98 series limit for data passed using the call loaddata; command. Using dataread, Real*8, Real*16 and character*8 data are supported for vectors or 1 D arrays of data. The related call csv( ); command will read Excel csv files. call dataread( ); Option # 1 uses the formated flat file. It will read the variable names in and optionally character*8 data. Unless set otherwise, real*8 is assumed. Example call dataread(:file 'mydata.dat'); Required :file => name of file or :unit => unit number Optional :precision key => real8 => saves as real*8 real16 => saves as real*16 vpa => saves as vpa data. default is real*8 Alternatives are :real8, :real16 and :vpa :real8 => saves as real*8. This is the default :real16 => saves as real*16 :vpa => saves as VPA :saveasvector => Default is :saveasarray :saveasarray => Save as 1 D array. Default :character namelist( ) => Sets character variables This is not allowed if there are :precision VPA data. :noob => Sets number of observations :nvar => Sets number of variables :means => Creates %vnames %means :debug => Place as first key word to output reading of data to assist in the debug process. This is usually not needed. Format of format flat file if :noob and :nvar not supplied. card 1 # obs # var card 2 variable names limited to 8 characters card 3 data Format of format flat file if :noob and :nvar are supplied. card 1 variable names limited to 8 characters card 3 data ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Option # 2 uses a read of a matrix into names. Required :file ' ' => name of file or :unit i => Unit number :names namelist(x y ) :noob int => Sets # of observations to read. If :file used, will read from top. If unit set can read in the middle of the file. This option must be used with caution. Optional :nocols i => Sets maximum number of cols read. Default = 512. This option is usually not needed. When used, inspect input closely to make sure all data read correctly. Up to 9000 cols can be read. :precision key => real8 => saves as real*8 real16 => saves as real*16 vpa => saves as vpa data. default is real*8 Alternatives are :real8, :real16 and :vpa :real8 => saves as real*8. This is the default :real16 => saves as real*16 :vpa => saves as VPA :character namelist( ) => Sets character variables. This is not allowed if there is vpa data. :format '( )' => valid fortran format. Format cannot be used if there is vpa data or a read in being made into real*16 or character data. The purpose of this option is to allow jumping columns if needed. If the :format option is used then :nocols option is not supported. :saveasvector => Default is :saveasarray :saveasarray => Save as 1 D array. Default :means => Creates %vnames %means Example # 1 Three simple data reads. Note that datacards automatically places data on unit 4 which does notr have to be opened anbd should nto be closed. /; Shows a simple read b34sexec matrix; datacards; 3 4 x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 b34sreturn; call dataread(:unit 4); call names; call tabulate(x1 x2 x3 x4); b34srun; /; Here :noob and :nvar are set, first card not needed b34sexec matrix; datacards; x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 b34sreturn; call dataread(:unit 4 :precision real8 :noob 3 :nvar 4); call names; call tabulate(x1 x2 x3 x4); b34srun; /; Reading from a file b34sexec matrix; call open(77,'test2.txt'); c=c1array(4,12: ' x1 x2 x3 ' ' 11. 22. 33.' ' 44. 55. 66.' ' 77. 88. 99.'); /; build test file do i=1,4; call write(c(i,),77); enddo; call close(77); /; Now read test file call dataread(:file 'test2.txt' :noob 3 :nvar 3); call tabulate(x1,x2,x3); b34srun; Example # 2 More Complex Examples of dataread /; b34sexec options debugsubs(readob_vpa, /; b34smatdataread); b34srun; /; Shows three reads b34sexec matrix; datacards; 3 4 x1 x2 x3 x4 11 22 33 44 55 66 77 88 99 10 11 12 3 4 y1 y2 y3 y4 11 22 33 joe 55 66 77 sam 99 10 11 tom b34sreturn; call dataread(:unit 4 :precision real8); call names; call tabulate(x1 x2 x3 x4); /; Testing means call rewind(4); call dataread(:unit 4 :precision real8 :means); call print(%means,%vnames); /; Read next 4 obs call dataread(:unit 4 :character namelist(y4)); call tabulate(y1 y2 y3 y4); call rewind(4); call names(all); /; now reading first 4 in real*16 oldx1=x1; oldx2=x2; oldx3=x3; oldx4=x4; call rewind(4); call print('Reading data into real*16':); call print('_________________________':); call dataread(:unit 4 :precision real16 :means); call tabulate(x1 x2 x3 x4); call print(%means,%vnames); call names(all); call rewind(4); call names(all); /; Reading in vpa oldx1=x1; oldx2=x2; oldx3=x3; oldx4=x4; call print('Reading data into VPA':); call print('_____________________':); call dataread(:unit 4 :precision vpa :means ); call tabulate(x1 x2 x3 x4); call print(%means); call names(all); b34srun; /; Using Option # 2 b34sexec matrix; datacards; 11 22 33 44 55 66 77 88 99 10 11 12 11 22 33 joe 55 66 77 sam -9 -8 -7 tom b34sreturn; /; /; doing two reads; /; call dataread(:unit 4 :names namelist(x1 x2 x3 x4) :precision real8 :noob 3); call tabulate(x1 x2 x3 x4); call dataread(:unit 4 :character namelist(y4) :names namelist(y1 y2 y3 y4) :noob 3); call tabulate(y1 y2 y3 y4); b34srun; /; Create a file and read it back b34sexec matrix; call echooff; noob=100; nvar=3; x=rn(array(noob,nvar:)); call print(x); call open(77,'test.txt'); do i=1,noob; call write(x(i,),77); enddo; call close(77); call dataread(:file 'test.txt' :noob noob :names namelist(x1 x2 x3) :precision real16); call tabulate(x2,x2,x3); call print(mean(x(,1)), mean(x(,2)), mean(x(,3))); call print(mean(x1 ), mean(x2 ), mean(x3 )); call open(77,'test2.txt'); c=c1array(3,12: ' 11. 22. 33.' ' 44. 55. 66.' ' 77. 88. 99.'); do i=1,3; call write(c(i,),77); enddo; call close(77); call dataread(:file 'test2.txt' :noob 3 :format '(3f4.0)' :names namelist(x1 x2 x3) :precision real8); call tabulate(x1,x2,x3); b34srun; Example # 3 using dataread to make a mac file. The only limit is the B34S data step limit of 98. This example gets around 80 col limit on input. In matrix, there is effectively no variable limit. Note: call catname( ) needed if in the namelist there is a 8 character name. In this application a blank is added. b34sexec matrix; call load(catname :staging); call dataread(:file 'murder.raw' :noob 153 :nvar 13 :means :nocols 700 :names namelist(id state year mrdrte exec unem d90 d93 cmrdrte cexec cunem cexec_1 cunem_1) :character namelist( stateabb state) ); call setlabel(id ,'state identifier '); call setlabel(state ,'postal code '); call setlabel(year ,'87, 90, or 93 '); call setlabel(mrdrte ,'murder rate '); call setlabel(exec ,'total executions, past 3 years'); call setlabel(unem ,'annual unem. rate '); call setlabel(d90 ,'=1 if year == 90 '); call setlabel(d93 ,'=1 if year == 93 '); call setlabel(cmrdrte,'mrdrte - mrdrte[_n-1] '); call setlabel(cexec ,'exec - exec[_n-1] '); call setlabel(cunem ,'unem - unem[_n-1] '); call setlabel(cexec_1,'cexec[_n-1] '); call setlabel(cunem_1,'cunem[_n-1] '); call print(%means); call catname(%vnames,' ',%vnames,0); call makedata(argument(%vnames) :file 'wood.mac' :heading 'murder data' :member murder :add); DATAVIEW View a Series Under Menu Control subroutine dataview(x,'xname'); /$ /$ Views data series x under User Control /$ x => series /$ xname => x series name /$ Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(dataview); call load(data_acf); call load(cperiod); call dataview(gasout,namelist(gasout)); b34srun; DES - Code / decode. This command should be regarded as experimental. call des(ch1,ch2,key,task); ch1 - in text. Must be character*8 or Character*1 of up to 16 characters. ch2 - out text key - up to 16 characters task - = 0 => code = 1 => decode Text must be in hex form. See chtohex and hextoch. b34sexec matrix; /$ 12345678901234567890123456789012 call character(line1, 'This is a test of the system '); call character(line2, 'This is line # 2 of code test'); call chtohex(line1,hexline1); call chtohex(line2,hexline2); call print(hexline1,hexline2); hexline1=c1array(4,16:hexline1); hexline2=c1array(4,16:hexline2); call print(hexline1,hexline2); in=catrow(hexline1,hexline2); call print(in); call character(key,'0101010101010101'); out=c1array(norows(in)*2,nocols(in):); in=transpose(in); do i=1,nocols(in); call des(in(,i),work ,key,0); out(,i)=work; enddo; call print(out); test=c1array(nocols(in),norows(in):); do i=1,nocols(in); call des(out(,i),work,key,1); call hextoch(work,work2); call print(work2); test(i,)=work2; enddo; call names(all); call print(test); i=integers(1,nocols(test)/2); newtest=test(,i); call print(c1array(norows(newtest)* nocols(newtest):transpose(newtest))); b34srun; Test cases in DES. DEBUGSUBS Internal debug capability call debugsubs(:routine1 :routine2); This is not intended for user unless given instructions by developer of B34S. The purpose of this command is to turn on internal write statements intended for developers / debugging. An example of possible use ot users is: call debugsubs(:byplin); which will make a duplicate copy of output in file debug that will reflect just where the job stopped. call debugsubs is the matrix command version of b34sexec options debugsubs(routine1 routine2); b34srun; The debug can be terminated by passing a dummy argument such as call debugsubs(:stopdebug); DESCRIBE Calculate Moment 1-4 and 6 of a series call describe(x); call describe(x :print); Variables created are: %mean = mean %sd = small sample SD %sk = skewness %c4 = kurtosis %c6 = 6-th order cumulant %max = maximum %min = minimum %median = median %q1 = First Quartile %q3 = Third Quartile %jb_test = Jarque-Bera (1987) Normality test %jb_prob = Probability of Jarque-Bera Test %sk_adj = Adjusted Skewness %c4_adj = Adjusted Kurtosis %sk_z = z value for Skewness %sk_prob = Probability of accepting Skewness %c4_z = z value for Kurtosis %sk_prob = Probability of accepting Kurtosis %freq = frequency = 0 if not a time series %timeb = timebase %times = timestart Note: If missing data is found, the command returns. %sd = sqrt(sum(x(i)**2)/(n-1) - sum(x)**2) %sk = sum((x(i)-mean(x))**3.)/(N*sd**3.) %c4 = sum((x(i)-mean(x))**4.)/(N*sd**4.)-3.0 %c6 = sum((x(i)-mean(x))**6.)=15.0*%c4 -10*%sk*%sk-15. %sk_adj = ((n**2)/((n-1)*(n-2))*(m3/s**3) ((n+1)*m4 -3.(N-1)*m2*m2)/s**4 %c4_adj = ((n**2)/((n-1)*(n-2)*(n-3))* ((n+1)*m4 -3.(N-1)*m2*m2)/s**4 %jb_test = N*( ((%sk_adj*%sk_adj)/6.) + ((%c4_adj*%c4_adj)/24.)) %jb_prob = Chisq probability with DF = 2 of %jb_test %sk_z = %sk_adj*sqrt(((n-1)*(n-2))/(6*n)) %c4_z = %c4_adj*sqrt(((n-1)*(n-2)*(n-3))/ (24*n*(n+1)) Note: %sk_adj and %c4_adj and %jb_test are the same as Rats and are based on formulas from Kendall & Stuart (1958) This command works for real*8 data. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; x=rn(array(1000:)); call describe(x :print); call describe(gasin :print); call describe(gasout :print); b34srun; DUD Derivative Free Nonlinear Estimation call dud(xvar,yvar,beta,r,f,sse,seb,covb,corrb, iprint,iout); Routine based on SAS nonlinear routine dud in technical report a-102 page 8-9. Routine was implemented in Speakeasy April 1987 and in the B34S Matrix language June 1998. DUD provides derivative free nonlinear estimation. Its use is to study nonlinear estimation. For production use see NLLSQ and NL2SOL which are faster. DUD needs user subroutine resid. xvar = matrix of x variables - input yvar = left hand side variable vector - input beta = vector of initial guess on coefficients - input/output r = residual vector - output f = predicted variable vector - output sse = sum of squared residuals (sumsq(r)) - output seb = se's of the beta coefficients - output covb = covariance matrix of beta coefficients - output corrb = correlation matrix of beta coefficients - output iprint= 0 for no iteration print, =1 for iteration print - input iout = 0 for no output printing, =1 output will be given Test Cases: NLLS1, NLLS2, NLLS3 Example from NLLS1 b34sexec matrix cbuffer=10000; call echooff; call load(dud); call load(marq); program prob1; /$ /$ test marquardt method of nonlinear estimation /$ calls marquardt subroutine marq /$ user supplied resid and deriv /$ /$ imar=0 marquardt , =1 = dud /$ call message( 'enter=> deriv. method, Cancel=> deriv. free method', 'Estimation Options', itest); imar=0; if(itest.eq.23)imar=1; /$ get data call uspopdat; /$ initial values call free(deriv,resid,beta,r); resid=resid1 ; deriv=deriv1 ; /$ /$ rename routines on the fly /$ call subrename(resid); call subrename(deriv); call makeglobal(resid,deriv) ; beta(1)=3.9 ; beta(2)=.022 ; beta=vfam(beta) ; year=mfam(year) ; pop=mfam(pop) ; lamda=.1e-8 ; iprint=0 ; iout=1 ; /$call print('IMAR',imar); if(imar .eq. 0) call marq(year,pop,beta,r,f,sse,seb,covb,corrb, lamda,iprint,iout); if(imar .eq. 1) call dud(year,pop,beta,r,f,sse,seb,covb,corrb, iprint,iout); return; end; subroutine resid1(beta,f,r,sse,xvar,yvar); /$ /$ user supplied routine with model /$ sas tech report a-102 page 8-7 /$ f=vfam(beta(1)* exp(beta(2)*afam(xvar-1790.))); r=yvar-f; sse=sumsq(r); return ; end ; subroutine deriv1(der,f,beta,xvar); /$ /$ user routine to calculate derivatives /$ der=matrix(norows(f),norows(beta):); der(,1)=vfam(afam(f)/beta(1)); der(,2)=vfam(afam(xvar-1790.)*afam(f)); return; end; program uspopdat; /$ data from sas technical report page 9-2 year=dfloat(integers(179,197)); year=year*10. ; pop=array(:3.929 5.308 7.239 9.638 12.866 17.069 23.191 31.443 39.818 50.155 62.947 75.994 91.972 105.710 122.775 131.669 151.325 179.323 203.211 ); call tabulate(year pop); return; end; call print(prob1,resid1,deriv1); call prob1; b34srun; DELETECOL Delete a column from a matrix or array. call deletecol(x,jbegin); Deletes column of x at jbegin. The code for deleting more than one col is call deletecol(x,jbegin,number); The command call deletecol(x); deletes the last column. Note that jbegin and number are integer*4. DELETEROW Delete a row from a matrix or array. call deleterow(X,ibegin); Deletes row at ibegin. The code for deleting more than one row is: call deleterow(x,ibegin,number); The command call deleterow(x); deletes the last row. Note that ibegin and number are integer*4. DF Calculate Dickey-Fuller Unit Root Test. call df(x,d); Returns Dickey-Fuller Unit Root Test. x => Series to test d => DF test Added options: :df n => sets lag for DF test default = 1 :adf n => augmented DF test :adft n => augmented DF with trend :zform => uses z-form of test :print => Print value and significance Table options. x can contain more than one element. :table n => Generates prob value d for DF value x using "no constant assumption." :table2 n => Generates prob value d for DF value x using "constant assumption.". :table4 n => Generates table value d for DF value x using "constant plus trend." assumption. Automatic Variable Created %DFPROB - Probability of DF test. .05 => Cannot reject unit root at 95% .10 => Cannot reject unit root at 90% %DFVALUE - Critical values for 1% 5% 10%. Not calculated for :zform. Discussion: The .05 critical value for N=100 is -1.95. This suggests that if the value found was -2.0 (-1.95) we could reject (could not reject) a unit root at the 95% level. The .10 critical value is -1.61. Using this standard we can reject a unit root. The related command PP tests for a unit root using the Phillips Perron test. Notes: The DF and PP commands have a "table look up" routine that will return the Dickey Fuller values. The matrix.mac file DF1 uses Monti Carlo Methods to approximate this table if the x value passed is not .01, .025 .05 .10 .90 .95 .975 .99 Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call echooff; call print('Dickey Fuller Tests on Gasout'); call df(gasout,d :print); n=30; adf=array(n+1:); adft=array(n+1:); lag=array(n+1:); do i=0,n; call df(gasout,a1:adf i); call df(gasout,a2:adft i); j=i+1; adf(j)=a1; adft(j)=a2; lag(j)=dfloat(i); enddo; call print('Dickey-Fuller test',d); call tabulate(lag,adf,adft); b34srun; DF_GLS Elliot, Rothenberg-Stock DF_GLS Test call DF_GLS(x,lag1,notrend, trend, notrendx,trendx,iprint); Implements the Elliott-Rothenberg-Stock (1996) unit root test documented in "Efficient Tests for an Autoregressive Root" Econometrica 64(4): 813-836. See also "Introduction to Econometrics," By James Stock and Mark Watson, Addison Wesley New York 2003 page 549-550 x = series to test lag1 = Lag for DF part of test. Must be GE 1 notrend = > no trend test statistic trend = > trend test statistic notrendx = x smoothed without a trend trendx = x smoothed with a trend iprint = 2 to print steps and test, = 1 print test only Critical values: 10% 5% 1% No trend -1.62 -1.95 -2.58 Trend -2.57 -2.89 -3.48 Note: Command is a subroutine and needs to be loaded with: call load(df_gls); Example: b34sexec matrix; call load(df_gls); call print(df_gls); iprint=1; n=1000; x=rn(array(n:)); root=cusum(x); call graph(x); call graph(root); call echooff; do i=1,4; call print(' ':); call print('For lag ',i:); call print('Non unit root case':); call DF_GLS(x,i,notrend, trend, notrendx,trendx,iprint); call print(' ':); call print('----------------':); call print(' ':); call print('Unit root case':); call DF_GLS(root,i,notrend, trend, notrendx,trendx,iprint); enddo; b34srun; DISPLAYB Displays a Buffer contents call displayb(x); Will display the contents of X. X can be real*8, character*8, character*1, integer, real*4 or complex*16. This command is useful in looking at contents of database files etc. Alternative arguments are: call displayb(x,istart,iend) to display only bytes istart to iend Example: b34sexec matrix; call character(cc,'This is a test'); call displayb(cc); call character(cc2, 'This is a test with numbers 1 2 3 # $ % 7 && 8 &'); call displayb(cc2); * Put in reals we know what they are; x(1)=0.0; x(2)=1.0; * Hide an integer in a real; i1=1; i2=2; call ilcopy(4,i1,1,1,x,1,1); call ilcopy(4,i2,1,1,x,1,3); call displayb(x); b34srun; Example # 2: /$ /$ Shows moving a real*16 value in a real*8 work array /$ Uses a real*8 array to look at bits!! /$ b34sexec matrix; x=array(2:); y=10.0; y=r8tor16(y); yy=y; y=r8tor16(12.8); call print('is yy 10.? ',yy); call pcopy(2,pointer(y),1,pointer(x), 1,8); call pcopy(2,pointer(x),1,pointer(yy),1,8); call print('is yy 12.8.? ',yy); call displayb(x); call names(all); call displayb(yy); b34srun; DIST_TAB Distribution Table call dist_tab(x,10,q,qvalue,number,iprint); Gives distribution subroutine dist_tab(x,n,q,qvalue,number,iprint); /$ /$ x => input series /$ n => input # of quantile values /$ q => q /$ qvalue => qvalue /$ number => # in the group /$ iprint => NE 0 = print /$ /$ Built July 2003 /$ Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(dist_tab); call echooff; call describe(gasin :print); call dist_tab(gasin,20,q,gvalue,number,1); b34srun; DMFGET Load data from a DMF Save file call dmfget(:file 'file name' options); The dmfget command is the same as the getdmf command. Loads series from a b34s dmf file. Options: :file kk => default 'myruns.dmf' :browse => List what is in file :browsename => Lists names as well as members. :member kk => Sets member to read. Default => read first member; :series nn => Lists series to read. If series is not present, all series are loaded. nn is usually set as a namelist i. e. :series namelist(income, wages) :print => Print what is loaded. Note: the namelist %series contains the series loaded. Examples: call dmfget; call dmfget(:file 'myrun1.dmf'); call dmfget(:file 'myrun1.dmf' :series namelist(x,y,z)); call dmfget(:browse :print); Extended Examples: Loads series in DMF file b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2); call dmfget(browse); call cleardat; call dmfget; call names(all); b34srun; Loads series in a DMS File - Bigger esample b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call echooff; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); people=namelist(houston,diana,will,melissa,bobby); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2 %res %y %yhat people :file 'testdmf.dmf' :header 'RES (1972) data' :member test_dat :print); nrow=10; ncol=8; x=rn(matrix(nrow,ncol:)); call print(x); means=array(ncol:); do i=1,ncol; means(i)=mean(x(,i)); enddo; call print(means); call dmfput(x :print :file 'testdmf.dmf' :add :header 'Random Matrix Data'); call dmfget(:browse :browsename :file 'testdmf.dmf'); call cleardat; call dmfget(:file 'testdmf.dmf'); call names(all); call dmfget(:file 'testdmf.dmf' :member data1 :print); call names(all); test=catcol(m1col__1,m1col__3); call print(' ':); call print('mean(test(,1)),mean(test(,2))':); call print( mean(test(,1)),mean(test(,2))); test=catcol(lnq,lnl,lnk); call print(' ':); call print('mean(lnq),mean(lnl),mean(lnk)':); call print( mean(lnq),mean(lnl),mean(lnk)); b34srun; b34sexec options open('testdmf.dmf') unit=62; b34srun; b34sexec dmf infmt=formatted inunit=62$ browse listnames$ b34srun$ b34sexec data file('testdmf.dmf') dmfmember(test_dat) filef= dmf; b34srun; b34sexec list iend=10; b34srun; b34sexec data file('testdmf.dmf') dmfmember(data1 ) filef= dmf; b34srun; b34sexec list; b34srun; DMFPUT Place data in a DMF save file The dmfput command is the same as the makedmf command. call dmfput(x,y,z); Places x, y and z in a dmf formatted file with default name myruns.dmf. If the series are not the same length, they are padded with the b34s missing value. Only 1 d vectors, 1 d arrays and matrices can be passed. If X is a matrix, the dmf names are M1col_01. Only Real*8 or character*8 objects can be saved. The B34S command b34sexec dmf can further process these files and the b34sexec data command can selectively read from the DMF file. A SCA fsave file can be read into a dmf file with the dmf command. The main function of the DMF file is to allow subsets of data to be loaded in the b34s data step. An example is: b34sexec data filef=fdmf file('c:\mysd\mydata.dmf') dmfmember(crime)$ input x y z$ b34seend$ If MYDATA.DMF contained 80 series and the user wanted to load all 80 series without using an INPUT statement, the correct setup would be just to omit the INPUT statement. If the DMF file contained more than 98 series and the INPUT statement was omitted, only the first 98 series would be read. The capability of the dmf data file format may be extended to support more than real*8 and character*8 data at some future date. By use of the rtoch and chtor functions character data and data of other types can be hidden in a real*8 variable. This is illustrated in the following example: b34sexec matrix; x=array(5:1 2 3 4 5); call print(x); cx=rtoch(x); call names; newx=chtor(cx); call tabulate(x,newx); c=c8array(:'sue','diana','houston'); rc=chtor(c); call print(c,rc rtoch(rc)); b34srun; The default member name is DATA1. Optional arguments include: :print => will print a message :file 'file name here' Default myruns.dmf :header ch => sets up to 32 characters of data :member mname data1 :add If add is not present, the file is rewound. Add not implemented in release # 1 :Julian int Default = 1. If ii=-999999999; then :julian ii will turn off automatic time series. Note :julian -99 tries to perform a math calculation in error. :freq r8 Frequency. Default = 1. set :freq missing() to turn off automatic time series and in addition set julian to -999999999. :Comment Limit of 72 characters per row. Up to 99 rows. The filename must fit into 60 characters. Examples: call dmfput(x,y,z :file 'c:\test\my.dmf'); call dmfput(x,z :file 'my.dmf' :header 'run one' :member r1); Extended Examples: Loads series in DMF file b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2); call dmfget(:browse); call cleardat; call dmfget; call names(all); b34srun; Loads series in a DMS File - Bigger example b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call echooff; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); people=namelist(houston,diana,will,melissa,bobby); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2 %res %y %yhat people :file 'testdmf.dmf' :header 'RES (1972) data' :member test_dat :print); nrow=10; ncol=8; x=rn(matrix(nrow,ncol:)); call print(x); means=array(ncol:); do i=1,ncol; means(i)=mean(x(,i)); enddo; call print(means); call dmfput(x :print :file 'testdmf.dmf' :add :header 'Random Matrix Data'); call dmfget(:browse :browsename :file 'testdmf.dmf'); call cleardat; call dmfget(:file 'testdmf.dmf'); call names(all); call dmfget(:file 'testdmf.dmf' :member data1 :print); call names(all); test=catcol(m1col__1,m1col__3); call print(' ':); call print('mean(test(,1)),mean(test(,2))':); call print( mean(test(,1)),mean(test(,2))); test=catcol(lnq,lnl,lnk); call print(' ':); call print('mean(lnq),mean(lnl),mean(lnk)':); call print( mean(lnq),mean(lnl),mean(lnk)); b34srun; b34sexec options open('testdmf.dmf') unit=62; b34srun; b34sexec dmf infmt=formatted inunit=62$ browse listnames$ b34srun$ b34sexec data file('testdmf.dmf') dmfmember(test_dat) filef= dmf; b34srun; b34sexec list iend=10; b34srun; b34sexec data file('testdmf.dmf') dmfmember(data1 ) filef= dmf; b34srun; b34sexec list; b34srun; DMFMERGE Merge two DMF save files call dmfmerge( ); Will sort merge two dmf files into one dmf extract file. The command dmfmerge is the matrix command equivalent to the b34sexec merge command which. The advantage of dmfmerge is that it can be used in a loop inside the matrix command. The merge data will work for real*8 and character*8 data. The number of series in the merged dataset must be le 9999. The merge is done one of two ways: :obsmerge => Assumes the same number of observations in each file. Observation 1 in file 1 is merged with observation 1 in file 2. :sortmerge => Assumes the two files have a sort key which may or may not be a time variable. The two files are merged by this key. The number of observations in file2 must be le number of observations in file1. If the sort key is not found, missing data is placed in the file. The final number of observations is the number origionally in file1. For example assume file1 contain state monthly data for two years or 50*24 = 120 obs. File2 contains US data for two years or 24 obs. The combined file contains 120 obs of the state data that has been appended the correct US monthly data. Note: :obsmerge or :sortmerge must be supplied. :add => Adds to files already on FILE3. The default is to reuse the file. :print => Display info on the merge. :formatted => Sets output file to formatted. This is the default. The option :unformatted is not 100% supported at this time. :header ch => sets up to 32 characters of data :file1 n1 => Sets DMF name for file1. For example :file1 'file1.dmf' :file is as required parameter. :file2 n2 => Sets DMF name for file2. :file2 is a required parameter. :file3 n3 => Sets the DMF name for file3. If :add is set, file3 must exist. :file3 is required. :member1 m1 => sets member for file1. Default = first file. :member2 m2 => sets member for file2. Default = first file. :member3 m3 => sets member for file3. Default = checkpoint. Note: File1, file2 and file3 must NOT be the same file or in the same dataset.. :recode1 namelist( ) => sets variables to be recoded in file1. For example :recode1 namelist(oldvar1 newvar1 oldvar2 newvar2); :recode2 namelist( ) => sets variables to be recoded in file2. For example :recode2 namelist(oldvar1 newvar1 oldvar2 newvar2); :drop1 namelist => sets sets variables to drop in file1. For example :drop1 namelist(var1 var2) :drop2 namelist => sets sets variables to drop in file2. For example :drop2 namelist(var1 var2) :keep1 namelist => sets sets variables to keep in file1. For example :keep1 namelist(var1 var2) :keep2 namelist => sets variables to keep in file2. Note: If :drop1 is given :keep1 cannot be supplied. If :drop2 is given :keep2 cannot be supplied. :comment ' ' => sets comments for new file. Up to 72 characters supplied. :file1by v1 => sets merge variable from file1. :file2by v2 => sets merge variable from file2. Note file1by and file2by must be present if mergetype=sortmerge. In the exampel below note the use of %series that is created by the dmfget command to align series. The code to look at is: /; /; use align & %series to align data on the fly /; call print(%series); call align(argument(%series)); call print('Series now aligned to remove Missing Data':); call tabulate(id1 id2 lnq lnk x1 x2 x3 x4); call cleardat; call getdmf(:file 'file3.dmf' :print :series namelist(lnq lnl lnk)); call print(%series); The argument( ) function replaces the contents of %series or the variable values loaded directly into the align command. Example setup: call dmfmerge(:obsmerge :file1 'file1.dmf' :member1 data1 :file1by data1 :file2 'file2.dmf' :member2 air :file2by airn :file3 'test.dmf' :member2 master :comment 'Merging data from matrix' :print); Example: Illustrates obsmerge b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2 :file 'file1.dmf'); call makedmf(p k l m1 m2 :file 'file2.dmf'); call getdmf(:browse :file 'file1.dmf'); call dmfmerge(:file1 'file1.dmf' :member1 data1 :file2 'file2.dmf' :member2 data1 :file3 'file3.dmf' :member3 cdata :obsmerge :print); call cleardat; call getdmf(:file 'file3.dmf' :print); call names(all); call cleardat; call getdmf(:file 'file3.dmf' :series namelist(p k) :print); call print('mean of p ',mean(p)); call print('mean of p ',mean(k)); call names(all); b34srun; Example of Sortmerge b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); id1=dfloat(integers(1,norows(lnq))); call makedmf(id1 lnq,lnl,lnk,lnrm1,lnrm2 :file 'file1.dmf'); x1=rn(array(10:)); x2=rn(array(10:)); x3=rn(array(10:)); x4=rn(array(10:)); id2=dfloat(integers(norows(x1))); call tabulate(id1 id2 lnq lnk x1 x2 x3 x4); call makedmf(id2 x1 x2 x3 x4 :file 'file2.dmf'); call getdmf(:browse :file 'file1.dmf'); call getdmf(:browse :file 'file2.dmf'); call dmfmerge(:file1 'file1.dmf' :member1 data1 :file2 'file2.dmf' :member2 data1 :file3 'file3.dmf' :member3 cdata :file1by id1 :file2by id2 :sortmerge :print); call cleardat; call getdmf(:file 'file3.dmf' :print); call names(all); call tabulate(id1 id2 lnq lnk x1 x2 x3 x4); /; /; use align & %series to align data on the fly /; call print(%series); call align(argument(%series)); call print('Series now aligned to remove Missing Data':); call tabulate(id1 id2 lnq lnk x1 x2 x3 x4); call cleardat; call getdmf(:file 'file3.dmf' :print :series namelist(lnq lnl lnk)); call print(%series); b34srun; DODOS Execute a command string if under dos/windows. call dodos(' '); Works the same as call system(' ' ); but only works on Windows & DOS. Note: The form call dodos(' command'); should be used if "silent" operation is desired. If the command writes any output, the form call dodos('command',:); or call dodos('command':); must be used. If what is desired is for B34S to terminate and the program called to be active, the command call dodos('command',::); should be used. The key call dos( ) can be used in place of call dodos( ). Example /$ Matlab command file b34sexec options open('test.m') unit=77 disp=unknown; b34srun; b34sexec options clean(77); b34srun; b34sexec options copyf(4,77); pgmcards; x=rand(6) xi=inv(x); x*xi yy=[1 2 3 2 1] plot(yy) pause quit b34sreturn; b34srun; b34sexec options close(77); b34srun; b34sexec matrix; call system('start /w matlab /r test /logfile jj':); call copyout('jj'); b34srun; DO_SPEC Display Periodogram and Spectrum call do_spec(gasout,cc,weights); Will display the periodogram and Spectrum. subroutine do_spec(x,heading1,weights); /; /; Display Periodogram and Spectrum /; /; x = Input Series /; heading1 = Heading for series /; weights = Smoothing weights /; /; Note: Can be called alone or under dataview /; /; Graphs saved in Clip Board /; *********************************************** /; Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(do_spec); weights=array(:1 2 3 2 1); call character(cc,'Analysis of Gasout'); call do_spec(gasout,cc,weights); rr=rn(array(400:)); call character(cc,'Analysis of a Random Series'); call do_spec(rr,cc,weights); b34srun; DO2SPEC Display Periodogram and Spectrum file argument call do2spec(gasout,cc,weights,file); Will display the periodogram and Spectrum. subroutine do2spec(x,heading1,weights,file); /; /; Display Periodogram and Spectrum /; /; x = Input Series /; heading1 = Heading for series /; weights = Smoothing weights /; file = File name /; /; Note: Can be called alone or under dataview /; /; See also do_spec( ) /; *********************************************** /; Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(do_spec); weights=array(:1 2 3 2 1); call character(cc,'Analysis of Gasout'); call do2spec(gasout,cc,weights,'gas_spec.wmf'); rr=rn(array(400:)); call character(cc,'Analysis of a Random Series'); call do2spec(rr,cc,weights,'ran_spec.wmf'); b34srun; DOUNIX Execute a command string if under unix. call dounix(' '); Works the same as call system(' '); but only works on unix. Note: The form call dounix(' command'); should be used if "silent" operation is desired. If the command writes any output, the form call dounix('command',:); or call command('command':); must be used. If what is desired is for B34S to terminate and the program called to be active, the command call dounix('command',::); should be used. The key call unix( ) can be used in place of call dounix( ); DOT_PRODUCT Uses the built-in Fortran Function DOT_PRODUCT SUM call DOT_PRODUCT; Calling this command replaces the BLAS ddot and dsum with the Fortran internal dot_product and sum commands. This is done by "hot wiring" the BLAS ddot and dsum. Note that dot_product and sum are only utilized if the stride is 1. Note: calling DOT_PRODUCT will not disable REAL16ON etc. Example to illustrate speed depends on the problem /; /; Does not seem to make too much difference /; b34sexec options real16info ; b34srun; b34sexec matrix; call echooff; subroutine dotspeed(part1,part2,n,k); x=rn(matrix(n,n:)); y=rn(matrix(n,n:)); call compress; call dot_blas; call real16info; call timer(base1); do i=1,k; test=inv(x)*y; enddo; call timer(base2); part1=base2-base1; call compress; /; /; tests fortran dot_product and sum( ) /; call dot_product; call real16info; call timer(base3); do i=1,k; test=inv(x)*y; enddo; call timer(base4); part2=base4-base3; return; end; nn=12; size =array(nn:); blas =array(nn:); dot_p =array(nn:); do i=1,nn; size1=100*i; call dotspeed(part1,part2,size1,1); size(i) =size1; blas(i) =part1; dot_p(i)=part2; enddo; call print(' '); call tabulate(size,blas,dot_p); b34srun; Obs SIZE BLAS DOT_P 1 100 0.000 0.1562E-01 2 200 0.6250E-01 0.4688E-01 3 300 0.2500 0.2500 4 400 0.6719 0.7031 5 500 1.562 1.516 6 600 2.812 2.766 7 700 4.484 4.484 8 800 6.688 6.672 9 900 9.562 9.594 10 1000 13.17 13.14 11 1100 17.53 17.64 12 1200 22.81 22.67 DOT_BLAS Uses the BLAS Unrolled loop routines call DOT_BLAS; See example in call dot_product. calling this command uses the blas ddot command. DQDAG - Integrate a function using Gauss-Kronrod rules call dqadg(f x :name test :lower 0.0 :upper 2.0 :errabs 0.0 :errrel .001 :irule 1 :maxsub 500 :print); Integrates a function using Gauss-Kronrod Rules. Required: f = function value x = Integration variable test = program name :lower a => sets lower bound of integration :upper b => sets upper bound of integration Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :rule i => Sets Gauss-Kronrod rule. Default=2 If function has a peak singularity use :rule 1, if function is oscillatory, use :rule 6 1 => 7-15 points 2 => 10-21 points 3 => 15-31 points 4 => 20-41 points 5 => 25-51 points 6 => 30-61 points :maxsub i => sets # of subiterations allowed. Default=500. Variables Created: %result = value of integral %error = error estimate %alist = list on left endpoints %blist = list of right endpoints %rlist = area in the endpoints. %elist = error estimates by regions Note: This command uses IMSL routine DQDAG Example: b34sexec matrix; program test; f=x*dexp(x); return; end; call print(test); call echooff; do i=1,6; call dqdag(f x :name test :lower 0.0 :upper 2.0 :errabs 0.0 :errrel .001 :rule i :maxsub 500 :print); enddo; b34srun; DQDNG - Integrate a smooth function using a nonadaptive rule. call dqdng(f x :name test :lower 0.0 :upper 2.0 :errabs 0.0 :errrel .001 :print); Integrates a smooth function using a nonadaptive rule Required: f = function value x = Integration variable test = program name :lower a => sets lower bound of integration :upper b => sets upper bound of integration Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 Note: This command uses IMSL routine DQDNG. It may not work well, if so try dqdags. Variables Created: %result = value of integral %error = error estimate Example: b34sexec matrix; program test; f=x*dexp(x); return; end; call echooff; call dqdng(f x :name test :lower 0.0 :upper 2.0 :errabs 0.0 :errrel .001 :print); b34srun; DQDAGI - Integrates over a infinite/semi-infinite interval. call dqdagi(f x :name test :lower 0.0 :upper 0.0 :errabs 0.0 :errrel .001 :maxsub 500 :print); Integrates a function over infinite/semi-infinite interval. Required: f = function value x = Integration variable test = program name :lower a => sets lower bound of integration :upper b => sets upper bound of integration Cannot have both upper lower. If lower => range = lower - Inf If upper => range = inf - upper If neither => range = -int - inf Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :maxsub i => sets # of subitervals used. Default=500. Variables Created: %result = value of integral %alist = list on left endpoints %blist = list of right endpoints %rlist = area in the endpoints. %elist = error estimates by regions %error = error estimate Note: maxsub determines an upper limit on # of intervals. This command uses IMSL routine DQDAGI Example: b34sexec matrix; program test; f=dlog(x)/(1.+(10.*x)**2.); return; end; call dqdagi(f x :name test :lower 0.0 :errabs 0.0 :errrel .001 :maxsub 500 :print); exact = -1.*pi()*dlog(10.)/20. ; error=%result-exact; call print('Exact ',exact:); call print('Error ',error:); call tabulate(%alist %blist %rlist %elist); b34srun; DQDAGP - Integrete a function with singularity points given call dqdagp(f x :name test :lower :upper 0.0 :errabs 0.0 :errrel .001 :breakp p :maxsub 500 :print); Integretes a function with singularity points given. Required: f = function value x = Integration variable test = program name :lower a => sets lower bound of integration :upper b => sets upper bound of integration :breakp p => sets vector of break points Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :maxsub i => sets # of subitervals used. Default=500. Variables Created: %result = value of integral %alist = list on left endpoints %blist = list of right endpoints %rlist = area in the endpoints. %elist = error estimates %error = error estimate Note: maxsub determines an upper limit on # of intervals. This command uses IMSL routine DQDAGP Example: program test; f=x**3.*dlog(dabs((x*x-1.0)*(x*x-2.0))); return; end; call dqdagp(f x :name test :breakp array(:1. dsqrt(2.)) :lower 0.0 :upper 3.0 :errabs 0.0 :errrel .001 :maxsub 500 :print); exact = 61.0*dlog(2.0)+77./4.*dlog(7.0) - 27.; error=dabs(%result-exact); call print('Exact ',exact:); call print('Error ',error:); call tabulate(%alist %blist %rlist %elist); b34srun; DQDAGS - Integrate a function with end point singularities call dqdags(f x :name test :lower :upper 0.0 :errabs 0.0 :errrel .001 :maxsub 500 :print); Integrates a function with end point singularities Required: f = function value x = Integration variable test = program name :lower a => sets lower bound of integration :upper b => sets upper bound of integration :breakp p => Sets vector of break points Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :maxsub i => sets # of subitervals used. Default=500. Variables Created: %result = value of integral %alist = list of left endpoints %blist = list of right endpoints %rlist = area in the endpoints. %elist = error estimates %error = error estimate Note: maxsub determines an upper limit on # of intervals Note: This command uses IMSL routine DQDAGS Example: b34sexec matrix; program test; f=dlog(x)/dsqrt(x); return; end; call dqdags(f x :name test :lower 0.0 :upper 1.0 :errabs 0.0 :errrel .001 :maxsub 500 :print); exact = -4.0; error=dabs(%result-exact); call print('Exact ',exact:); call print('Error ',error:); call tabulate(%alist %blist %rlist %elist); b34srun; DQAND - Multiple integration of a function call dqand(f x :name test :lower lower :upper upper :errabs 0.0 :errrel .001 :maxsub 500 :print); Estimates a multiple integral. A max of 20 integrals can be calculated. Required: f = function value x = Integration variable name. X must exist and be an array or vector of up to 20 elements. The size of x is n and sets the size expected for lower and upper. test = program name :lower a => sets lower bound of integration. A is a vector or array of size n. :upper b => sets upper bound of integration. B is a vector or array of size n. Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :maxsub i => sets # of evaluations allowed. I cannot be set > 256*n where n is # of elements in lower. Default = 256*n. Variables Created: %result = value of integral %error = error estimate Note: This command uses IMSL routine DQAND Example: b34sexec matrix; * This is a big problem. Note maxsub 100000 ; program test; f=dexp(-1.*(x(1)*x(1)+x(2)*x(2)+x(3)*x(3))); return; end; /$ We solve 6 problems. /$ As constant => inf and => pi()**1.5 lowerv=array(3:); upperv=array(3:); x =array(3:); call print(test); call echooff; j=integers(3); do i=1,6; cc=dfloat(i)/2.0; lowerv(j)=(-1.)*cc; upperv(j)= cc; call dqand(f x :name test :lower lowerv :upper upperv :errabs .0001 :errrel .001 :maxsub 100000 :print); call print('lower set as ',cc:); call print('results ',%result:); call print('error ',%error:); enddo; call print('Limit answer ',pi()**1.5 :); b34srun; DTWODQ - Two Dimensional Iterated Integral call dtwodq(f x y g h :name test1 test2 test3 :lower lower :upper upper :errabs 0.0 :errrel .001 :rule 1 :print); Estimates a 2 dimensional integral f= int(f(x,y))dy dx Required: f = function value x = outer integral y = inner integral g = inner integral lower bound g=g(x) h = inner integral upper bound h=h(x) test1 = program name for function. test1 creates f(x,y) test2 = program name for lower bound of inner integral test3 = program name for upper bound of outer integral :lower a => sets lower bound of outer integral :upper b => sets upper bound of outer integral Optional :errabs r1 => Sets absolute accuracy desired. Default = 0.0 :errrel r2 => Sets relative accuracy desired. Default = .001 :rule i => sets Gauss-Kronrod Rule 1 7-15 points 2 10-21 points 3 15-31 points 4 20-41 points 5 25-51 points 6 30-61 points For singular peak use 1 For oscillatory function use 6 Default = 6 :maxsub i => sets # of evaluations allowed. Should be set greater than or equal to 250. Variables Created: %result = value of integral %error = error estimate %alist = list of left endpoints %blist = list of right endpoints %rlist = area in the endpoints. %elist = error estimates Note: This command uses IMSL routine DTWODQ Warning. Be sure that the three programs supplied actually do what they are required to do. Example: /$ Fixed inner bounds test case first %b34slet prob1=1; %b34slet prob2=1; %b34sif(&prob1.eq.1)%then; b34sexec matrix; program test1; f=y*dcos(x+y*y); return; end; program test2; g=1.0; * g=(-2.)*x; return; end; program test3; h=3.0; * h=5.*x; return; end; call print(test1,test2,test3); call echooff; call dtwodq(f x y g h :name test1 test2 test3 :lower 0.0 :upper 1.0 :errabs .000 :errrel .001 :rule 6 :print); call print(' ':); call print('***************************':); call print('IMSL thinks result is -.514':); call print('results ',%result:); call print('error ',%error:); call tabulate(%alist,%blist,%rlist,%elist); b34srun; %b34sendif; /$ Problem # 2 %b34sif(&prob2.eq.1)%then; b34sexec matrix; program test1; f=y*dcos(x+y*y); return; end; program test2; * g=1.0; g=(-2.)*x; return; end; program test3; * h=3.0; h=5.*x; return; end; call print(test1,test2,test3); call echooff; call dtwodq(f x y g h :name test1 test2 test3 :lower 0.0 :upper 1.0 :errabs .001 :errrel .00 :rule 6 :print); call print(' ':); call print('***************************':); call print('IMSL thinks result is -.083':); call print('results ',%result:); call print('error ',%error:); call tabulate(%alist,%blist,%rlist,%elist); b34srun; %b34sendif; ECHOOFF Turn off listing of execution. call echooff; Turns off output to b34s file. By default all commands will echo. It is usually a good idea to turn off command echo inside a do loop unless there are problems to trap. Once a user subroutine, function or program is working correctly, it is a good idea to call echooff before calling the routine. If problems develop, then they can be easily trapped by commenting this call. Note: Matlab will echo any command without the ; and will not echo any command with a ;. This approach requires the developer to be taking out and putting in ; all over the place when problems are being tracked down. See related command call echoon; Note: The commands call echoon; x=10; call mysub(x); call echooff; will echo all commands inside the user routine mysub. The correct way to echo only x=10; is call echoon; x=10; call echooff; call mysub(x); ECHOON Turn on listing of execution. call echoon; Echos output in b34s output file. By default all commands will echo. It is usually a good idea to turn off command echo inside a do loop unless there are problems to trap. See related command call echooff; EPPRINT Print to log and output file. call epprint(x); Prints to both output and error units. See also eprint and print. EPRINT Print to log file. call eprint(x); Works the same as call print( ) but prints to the error unit. See also epprint and print. ERASE Erase a file call erase('c:\junk\*.out'); Will erase files. If a file cannot be deleted an error message is given. If ' ' is blank, there is no effect. Warning. Uses system calls. An open file can be deleted without an error message. Use this power commnd with caution. ESACF Extended Sample Autocorrelation Function call esach(x,np,nq); Estimates the Extended Sample Autocorrelation for series x Required: x => Series np => # of AR terms in table nq => # of MA terms in table Optional :print => Print the Table Variables Created: %ESACF => np+1 by nq+1 table where Columns represent MA and Rows represent AR terms: References Tiao, George O. and Ruey Tsay. "Multiple Time Series Modeling and Extended Sample Cross-correlations," Journal of Business and Economic Statistics, 1(1) (1983): 43-56. Tsay, Ruey and George Tiao. "Consistent Estimates of Autoregressive Parameters and Extended Sample Autocorrelation Function for Stationary and Non-stationary ARIMA Models," Journal of the American Statistical Association. 79 (1984) 84-96. Wei, William "Time Series Analysis: Univeriate and Multivariate Methods" Pearson (2006) page 128 Example: /; Tests IACF and esacf with B34S /; Uses Wei Data set (Lynx) b34sexec options ginclude('b34sdata.mac') member(wei_w7); b34srun; b34sexec matrix; call loaddata; call load(iacf :staging); call echooff; call print('+++++++++++++++++++++++++++++':); call print('Test Case From Wei page 130 ':); acf1=acf(lnpelts,20,se,pacf); call iacf(lnpelts,iacf1,10); call tabulate(acf1,pacf,iacf1:title 'ACF, PACF & IACF'); call esacf(lnpelts,5,9 :print); b34srun; /; /; Wei(2006) Page 132 /; B34S/SCA show slight Wei accuracy issues /; %b34slet dosca=1; b34sexec options ginclude('b34sdata.mac') member(bj_c); b34srun; b34sexec matrix; call loaddata; call load(iacf :staging); call echooff; call print('+++++++++++++++++++++++++++++':); call print('Test Case From Wei page 132 ':); call print( 'Note SCA and B34S get same answers but Wei differs':); acf1=acf(chem,20,se,pacf); call iacf(chem,iacf1,20); call tabulate(acf1,pacf,iacf1 :title 'ACF, PACF IACF'); call esacf(chem,5,9 :print); call print(' ':); call print('See what autobj gets ':); call print(' ':); call autobj(chem :print :nac 24 :npac 24 :autobuild ); call print(' ':); call print('Box-Jenkins-Reinsel (1994) page 196 ':); call print('Alternative Models ':); call autobj(chem :print :nac 24 :npac 24 :ar index(1 2)); call autobj(chem :print :nac 24 :npac 24 :dif index(2 1) :ma index(1 2)); b34srun; %b34sif(&dosca.ne.0)%then; b34sexec options open('sca.dat') disp=unknown unit(28)$ b34srun$ b34sexec options open('sca.cmd') disp=unknown unit(29)$ b34srun$ b34sexec options clean(28)$ b34srun$ b34sexec options clean(29)$ b34srun$ b34sexec pgmcall$ sca scafname=mydata$ pgmcards$ /$#==myrun --- these commands are required to load the b34s data. --- assign file 18. attrib access(read). external 'sca.dat'. call procedure is mydata. file is 18. --- iarima chem eacf chem acf chem iacf chem stop. return /$#== b34sreturn$ b34srun$ b34sexec options close(28)$ b34srun$ b34sexec options close(29)$ b34srun$ b34sexec options dounix('sca sca.cmd > sca.out') dodos('scaw32 10000 /f:sca.cmd /p:myrun /o:sca.out') $ b34srun$ b34sexec options npageout writeout('output from sca',' ',' ') copyfout('sca.out') dodos('erase sca.cmd','erase sca.out','erase sca.dat') dounix('rm sca.cmd','rm sca.out','rm sca.dat') $ b34srun$ EXPAND Expand an array call expand(oldcc,newcc,ibegin,iend); Expand an array of Character *1 data. oldcc - Character*1 string newcc - Character*1 string to be placed in ibegin - iend in oldcc moving old characters over. Elements in newcc from will be placed in oldcc from ibegin to iend. If length of newcc less than iend-ibegin+1, then blanks placed in file. Note: If just a replacement is needed then code such as /$ aabb at 5-8 b34sexec matrix; call character(cc,'This is a test'); call character(new,'aabb'); call print(cc); i=integers(1,4); j=i+4; cc(j)=new(i); call print(cc); b34srun; Will work. Example: b34sexec matrix; call character(cc,'This is a test'); call print(cc); call ilocatestr(cc,istart,iend); i=integers(istart,iend); subs=cc(i); call print(subs); call contract(cc,istart,iend); oldnewcc=cc; call print(cc); call character(new,'aaaissaa'); call expand(cc,new,1,8); call print(oldnewcc,cc); b34srun; For related commands see CONTRACT and the function EXTRACT. FILTER High Pass - Low Pass Filter using Real FFT call filter(xold,xnew,nlow,nhigh); Depending on nlow and nhigh subroutine filter can be a low pass or a high pass filter. A real FFT is done for a series. FFT values are zeroed out if outside range nlow - nhigh. Xnew is recovered by inverse FFT. FILTERC uses the complex FFT. FILTERC should be used in place of FILTER to avoid phase and gain loss. xold - input series xnew - filtered series nlow - lower filter bound nhigh - upper filter bound Routine built 2 April 1999. Use of filter requires the command call load(filter); Example: b34sexec matrix; /$ Uses FFT to High and Low Pass Random Series /$ /$ Illustrate with random numbers /$ call load(filter); n=500; test=rn(array(n:)); spec=spectrum(test,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random series'); call filter(test,newtest,1,200); spec=spectrum(newtest,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random after Low Pass'); call filter(test,high,201,500); spec=spectrum(high,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random after High Pass'); b34srun; FILTERC High Pass - Low Pass Filter using Complex FFT call filterc(xold,xnew,nlow,nhigh); Depending on nlow and nhigh filter can be a low pass or a high pass filter. Complex FFT is done for a series. FFT values are zeroed out if outside range nlow - nhigh. Xnew recovered by inverse FFT. FILTERC should be used in place of FILTER to avoid phase and gain loss. xold - input series xnew - filtered series nlow - lower filter bound nhigh - upper filter bound Routine built 2 April 1999. Use of filter requires the command call load(filterc); Example: b34sexec matrix; /$ Uses FFT to High and Low Pass Random Series /$ /$ Illustrate with random numbers /$ call load(filterc); n=500; test=rn(array(n:)); spec=spectrum(test,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random series'); call filterc(test,newtest,1,200); spec=spectrum(newtest,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random after Low Pass'); call filterc(test,high,201,500); spec=spectrum(high,freq); call graph(freq,spec :plottype xyplot :heading 'Spectrum of Random after High Pass'); b34srun; FLS Flexible Least Squares call fls(gasout gasin{1 to 6} gasout{1 to 6} :print); Calculates an fls regression. Discussion: The FLS regression selects a vector of coefficients that yield vector-minimun sums of squared residual measurement and dynamic errors conditional on the given observation. Simulation experiments demonstrate the ability of FLS to track linear, quadratic, sisusoidal, and regime shift motions in the true coefficients, despite noisy observations. Residual measurement errors are between y and yhat. Residual dynamic errors are discrepancies between estimated coefficient vectors at successive points in time. The "residual efficiency frontier" Define: %RSUBM = measurement error %RSUMD = dynamic error sum squared coefficient change) %RSUMOLS = Residual sum of squares from an OLS Model The user sets the penalty weight PWEIGHT that is used to determine the COST where COST = PWEIGHT * %RSEMD + %RSUMM As PWEIGHT increases the FLS coefficients are constrained not to change and %RSUBM approaches %RSUMOLS from below. OLS thus can be seen as aggregating the FLS coefficients to form an average with the result that their is a stiff penalty to be paid in terms of increases in measurement error. The OLS coefficients lie on the residual efficiency frontier. For detail see equation 6.1 in reference 1. This technique assumes the coefficients are either stable or are slowly evolving over time. References: 1. "Kalaba, R. and Leigh Tesfatsion, "Time-Varying Linear Regression Via Flexible Least Squares," Computers and Mathematics with Applications: Special Issue on System- Theoretic Methods in Econometric Modeling, (1989) 17,1215-1245. 2. Tesfatsion, Leigh and John Vietch. "U. S. Money Demand Instability: A Flexible Least Squares Approach," Journal of Economic Dynamics and Control (1990) 14, 151-173. Options: :print Print Results including variable means. This is the usual setting. :verify - Performs a test of how close the solution agrees with equations A14a, A14b and A14c of A15 of Kalaba-Tesfatsion (1989). Specifically define b(n) as the FLS coefficient vector for observations 1 to n. then 0 = sum(1,N)[transpose(X(n)*b(n)-y(n)]*X(n) for each n where 1 le n le N pweight[b(n+1)-b(n) = sum(1,n)[transpose(X(n)*b(n)-y(n)]*X(n) :pweight r1- Sets the weight on the dynamic error. As pweight increases, the coefficients are constrained to change less and less. Default = 1.0 :sample mask - Specifies a mask real*8 variable that if = 0.0 drops that observation. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better option is :holdout. :holdout n - Sets number of observations to hold out. This is useful for model validation purposes. :noint - Set no intercept. :savex - Save x data in %X. Note: :sample option cannot be used with :holdout. Variables Created: %YVAR => Name of left hand variable. %Y => Dependent Variable Data. %X => X matrix if :savex is supplied. %NAMES => Names of exogenous variables as entered. %LAG => Lag of exogenous variable as entered. %NVAR => Number of right hand variables %NOB => # of Observations in Model %BFLS => nob k vector of FLS values. %BOLS => OLS Solution. vector of k %BOLSE => Estimation of OLS from solution of equation 6.2 proved in Theorem 6.2 of reference 1. This should match %BOLS closely %RSUMBM => Measurement error %RSUMBD => Dynamic Error %COST => = (PWEIGHT*%RSUMBD) + %RSUMBM %UW_COST => = %RSUMBD + %RSUMBM %A_FLSB => = average of the FLS beta coefficients. If this is close to the OLS beta, then their no instability. The larger pweight value the closer %A_FLS_B is to the %BOLS. %SE_FLSB => = Standard error of %A_FLSB. The routine FLSGRAPH available in matrix2.mac is often a good way to display results. The call is just call flsgraph; Program flsgraph calls the lower level subroutine flsgrf part of which is shown below. program flsgraph; call flsgrf(%bfls,%bols,%names,%lag,%x,%y,%resid, %yhatfls,%resfls); return; end; subroutine flsgrf(bfls,bols,names,lags,x,y,resid, yhatfls,resfls); /; /; Graph FLS Estimated Coefficients /; /; Usually called from program flsgraph /; /; bfls => FLS Coefficient matrix. See %bfls /; bols => OLS Coefficient vector. See %bols /; names => Names Vector. See %names /; lags => Lag Vector. See %lag /; x => x matrix. Set :savex and use %x /; y => y vector. Use %y /; resid => OLS Residual. Use %resid /; /; out: /; /; yhatfls=> Yhat from FLS /; resfls => res from FLS /; /; The files FLS1____.wmf ... FLSk____.wmf save /; coefficient plots. /; Residuals are saved in file fls_ols_residuals.wmf /; /; Experimental Display Routine built 14 September 2010 The FLS Frounter can be calculated for 2D and 3 plotting with the routine flsfront. The complete routine is listed below: subroutine flsfront(lower,upper,inc,head,y,x,iscale,me,de,w, tmat); /; /; FLS Frontier /; /; lower => lower bound on weight /; upper => upper bound on weight (closed to OLS) /; inc => Incrument between weights /; head => Heading for Graph /; y => Set to %y from call fls( ) /; x => Set to %x from call fls( :savex) /; iscale => =0 map raw data /; =1 map scaled de data /; =2 map scaled me data /; =3 map both scaled me data and de data /; me => calculated Measurement Error data /; de => calculated Dynamic Error Data /; w => Weights used to calculate DE or ME /; tmat => Matrix of t values to test for Coef. stability /; given cost setting /; /; Experimental. Built 15 September 2010 by Houston H. Stokes /; Mods 24 September 2010 /; w=grid(lower,upper,inc); nn=norows(w); me=array(nn:); de=array(nn:); tmat = array(nn,nocols(x):); trow = array(nocols(x):); do i=1,nn; call fls(y x :pweight w(i) :noint); me(i)=%rsumbm; de(i)=%rsumbd; trow=divide(%a_flsb,%se_flsb); tmat(i,)=dabs(trow); enddo; xx ='Dynamic Error'; yy ='Measurement Error'; if(iscale.eq.1.or.iscale.eq.3)then; yy='Scaled Measurement Error'; me=(me/mean(me)); me=me-min(me); endif; if(iscale.eq.2.or.iscale.eq.3)then; xx='Scaled Dynamic Error'; de=(de/mean(de)); de=de-min(de); endif; ntick=4; demin=min(de); demax=max(de); ri1=(demax-demin)/dfloat(ntick-1); memin=min(me); memax=max(me); ri2=(memax-memin)/dfloat(ntick-1); call graph(de me :plottype xyplot :heading head :ylabelleft yy :xlabel xx :nocontact :setxscale array(:demin,ri1) :nxticks ntick :setyscale array(:memin,ri2) :nyticks ntick :pgborder :file 'FLS_frontier.wmf'); call graph(de me w :plottype contourc :heading head :ylabelleft yy :xlabel xx :zlabelleft 'Weight' :file 'FLS_3dfrontier.wmf' :angle 10. :d3axis /; :pspaceon :d3border :rotation 180.0 ); call graph(tmat :plottype meshstepc :heading head :ylabelleft 'Coefficient #' :xlabel 'Model Number' :zlabelleft 'Abs (t)' :file 'FLS_3d_stability.wmf' :angle 10. :d3axis /; :pspaceon :d3border :rotation 0.0 ); return; end; Routines flsfront, flsgraph and flsgrf are loaded with call load(flsgraph); Example: %b34sif(&tcase1.ne.0)%then; b34sexec data heading('FLS test data') noob=30; build y x1 x2 true1 true2; /; do i=1,ncap /; ai=dfloat(i) /; pi=(datan(1.0d+00))*4.0d+00 /; trueb(1,i)=.5d+00*dsin((2.0d+00*pi/30.0d+00)*ai) /; trueb(2,i)=dcos((2.0d+00*pi/30.0d+00)*ai) /; enddo /; /; x(1,1)=1.0d+00 /; x(2,1)=1.0d+00 /; /; do i=2,ncap /; ai=dfloat(i) /; x(1,i)=dsin(10.0d+00+(ai))+.01d+00 /; x(2,i)=dcos(10.0d+00+(ai)) /; enddo gen x1=sin(10.0+kount())+.01; gen x2=cos(10.0+kount()); gen true1=.5*sin(kount()*(2.*timespi(1.)/30.)); gen true2= cos(kount()*(2.*timespi(1.)/30.)); /; gen if(kount().eq.1)then; gen x1=1.; gen x2=1.; gen endif; gen y=x1*true1+x2*true2+(0.0*rn()); /; b34srun; /; b34sexec list; b34srun; /; b34sexec reg noint; model y = x1 x2; b34srun; b34sexec matrix; call echooff; call loaddata; call load(flsgraph); call olsq(y x1 x2 :noint :print ); call fls( y x1 x2 :noint :savex :print); call flsfront(.2,10.,1.,'Test of Basic Model 1',%y,%x, 0,%me,%de,%w,%tmat); call flsgraph; /; call tabulate(%y %yhat %yhatfls %resid %resfls); /; call print(%bfls); b34srun; %b34sendif; /; /; good model /; %b34sif(&tcase2.ne.0)%then; b34sexec matrix; call echooff; call load(flsgraph); call print('Good Model'); n=500; k=3; x=rn(matrix(n,k:)); beta=vector(k:)+10.0; y=x*beta + (10.*rn(vector(n:))); call olsq(y x :noint :print); call fls( y x :noint :print :savex); call flsfront(.2,90.,.1,'Test of Basic Model 2',%y,%x, 0,%me,%de,%w,%tmat); call flsgraph; call tabulate(%y %yhat %yhatfls %resid %resfls); call print(%bfls); b34srun; %b34sendif; %b34sif(&tcase3.ne.0)%then; b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call echooff; call load(flsgraph); time=time-1929.; call olsq(lnq lnl lnk lnrm2 time :print ); call fls(lnq lnl lnk lnrm2 time :print :pweight 1.e+0 :savex); call flsfront(.2,90.,.1,'Test of Basic Model 3',%y,%x, 3,%me,%de,%w,%tmat); call flsgraph; /; call tabulate(%y %yhat %yhatfls %resid %resfls); /; call print(%bfls); b34srun; %b34sendif; %b34sif(&tcase4.ne.0)%then; b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call echooff; call loaddata; call load(flsgraph); n=6; call olsq(gasout gasout{1 to n} gasin{1 to n} :print); call fls(gasout gasout{1 to n} gasin{1 to n} :print :pweight 1.e+0 :savex); call flsfront(.2,10.,.1,'Test of Basic Model 4',%y,%x, 3,%me,%de,%w,%tmat); call flsgraph; call tabulate(%y %yhat %yhatfls %resid %resfls); call print(%bfls); b34srun; %b34sendif; FLSGRAPH Graph FLS Results call flsgraph(bfls,bols,names); Will display FLS results. For more detail see help for FLS Example: b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call echooff; call loaddata; call load(flsgraph); n=6; call olsq(gasout gasout{1 to n} gasin{1 to n} :print); call fls(gasout gasout{1 to n} gasin{1 to n} :print :pweight 1.e+0 :savex); call flsfront(.2,10.,.01,'Test of Basic Model 4',%y,%x); call flsgraph; call tabulate(%y %yhat %yhatfls %resid %resfls); call print(%bfls); b34srun; FLSFRONT Will calculate and display the FLS Frontier. call flsfront(lower,upper,inc,head,y,x); The complete listring is: subroutine flsfront(lower,upper,inc,head,y,x); /; /; FLS Frontier /; /; lower => lower bound on weight /; upper => upper bound on weight (closed to OLS) /; inc => Incrument between weights /; head => Heading for Graph /; y => Set to %y from call fls( ) /; x => Set to %x from call fls( :savex) /; /; Experimental. Built 15 September 2010 by Houston H. Stokes /; g=grid(lower,upper,inc); nn=norows(g); me=array(nn:); de=array(nn:); do i=1,nn; call fls(y x :pweight g(i) :noint); me(i)=%rsumbm; de(i)=%rsumbd; enddo; call graph(de me :plottype xyplot :heading head :nocontact :pspaceon :pgborder :file 'FLS_frontier.wmf'); call graph(de me g :plottype contour3 :heading head :d3axis :ylabelleft 'Measurement error' :xlabel 'Dynamic error' :zlabelleft 'Weight' :file 'FLS_3dfrontier.wmf' :angle 10. :d3axis :pspaceon :d3border :rotation 180.0); return; end; Example: b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call echooff; call loaddata; call load(flsgraph); n=6; call olsq(gasout gasout{1 to n} gasin{1 to n} :print); call fls(gasout gasout{1 to n} gasin{1 to n} :print :pweight 1.e+0 :savex); call flsfront(.2,10.,.01,'Test of Basic Model 4',%y,%x); call flsgraph; call tabulate(%y %yhat %yhatfls %resid %resfls); call print(%bfls); b34srun; FPLOT Plot a Function call fplot(inline('dcos(dsqrt(x**2.+y**2.))'), :args x y :rangex array(:-10.,10.) :rangey array(:-10.,10.) ); This command has not been implemented in this release. FPRINT - Formatted print facility. call fprint(:clear :display rr '(g48.32)' :print); Advanced printing capability with format control. The casual matrix programmer usually does not need this command and can use the more general call print( ); :clear => clear buffer :col n => go to col 10 :string ' ' => pass a string to buffer (As of June 2005 :string and :display work the same for character*8 and character*1. See use notes below. :display object fmt => display object using optional format. Limited to a 132 line. Types supported are: real*8 complex*16 real*4 integer*4 integer*8 real*16 character*8 character*1 complex*32 fm fp im ip zm zp :print => prints the buffer. Does not clear buffer!! Sections of buffer can be cleared by passing a blank string. :save bname => saves buffer :cr n => blank lines :unit ii => Sets unit for output. If unit not present then usual output unit is assumed. Notes: The line a='le 8 '; creates a character*8 variable while a='more than 8 characters here'; creates a character*1 variable. The command a=c1array(:'aa'); places aa in a character*1 array. Hence for best results it is a good idea to code :col i :string c1array(:':') rather than :col i :string ':' for more control. Example: b34sexec matrix; r =dsqrt(110.); ii=202; name='Diana'; call fprint(:clear :col 10 :string 'At 10' :col 20 :display r '(g16.8)' :col 40 :string 'At col 40' :print :col 60 :string 'Added string at 60' :print :clear :string 'String at 1' :print :col 40 :string 'Added at 40' :col 70 :string name :print :cr 2); b34srun; Notes: The internal print buffer of 132 lines is saved between calls. This allows fprint to be a way to format a line!! In first call :clear should be used. b34sexec matrix; call echooff; call fprint(:clear :col 1 :string 'Mars Results after Stepwise Elimination'); call fprint(:print); call fprint(:save jj); call print(jj); call ialen(jj,ii); call print('len was ',ii); jjj=integers(1,ii-11); less=jj(jjj); call print(less); b34srun; /; Shows building a vector of names /; The vector of 246 names can be passed to a routine with /; the argument(cc) function b34sexec matrix; cc=c8array(246*2:); /; shows moving from character*1 to character*8 using /; fprint buffer /; tt is longer than needed but we pick off just first /; element j=1; do i=1,246; call fprint(:clear :col 1 :string 'X000'); if(i.le.9) call fprint(:col 4 :display i '(i1)'); if(i.le.999.and.i.ge.10) call fprint(:col 3 :display i '(i2)'); if(i.gt.99) call fprint(:col 2 :display i '(i3)'); call fprint(:save cc1); tt=c8array(:cc1); cc(j)=tt(1); j=j+2; enddo; call print(cc); b34srun; FREE Free a variable. call free(x); Frees X. The free command will free at the local level and above. If the variable is defined at both the global level and the local level, it is freed at the local level. To free at the global level use the form: call free(x:); Multiple series can be listed. Example: b34sexec matrix; n=4; x=rn(matrix(n,n:)); pdx=transpose(x)*x; call names; call free(n:); call names(info); call makeglobal(pdx); call names(info); r=pdfac(pdx); call print(pdx,r); call makelocal(pdx); call names(info); r=pdfac(pdx); call print(pdx,r); pdx(1,1)=.9999; call names; call print(pdx,'We now free at the local level'); call free(pdx); call names(info); call print('We now free at the global level'); call free(pdx:); call names(info:); b34srun; FORMS - Build Control Forms The FORMS options under the MATRIX command allows access to the Interacter low-level menu generation forms routines. This command is NOT intended for the general user. To make use of this command the user has to license the Interacter Software system and obtain the supporting manuals. The forms facility allows the B34S developer to have access to a general menu writting facility. The general B34S user uses the forms facility to interactively run MATRIX commands. call forms(:start ); call forms(:cont ); call forms(:final ); -------------------------------------------------------- :start sentence -------------------------------------------------------- Required on :start as the first argument :formdefine key index(ifield) index(ix) index(iy) index(iwidth) index(itype) key type of form W single form S Full screen T tabbed ifield => field numbers ix => array of field col positions iy => array of field row positions iwidth => array of field widths itype => array for field types 1 unprotected string 2 unprotected integer 3 unprotected real 4 cycling 5 push button 6 unprotected double 7 vertical menu 8 unprotected long string 9 check box 10 check box discription add 1000 for protected fields or :formload filename key W => Form in a window S => Form Full Screen T => Form Tabbed ----------------------------- Options on :start ----------------------------- :formdefinetabs key array(:labels) index(limitf) index(limitb) key T => show tabs on top R => show tabs on right B => show tabs on bottom labels char*8 array of size ntabs limitf integer array of last field identifiers limitb array of last box identifiers :formdefinebox index(iboxnum) index(ix) index(iy) index(iw) iboxnum => array of box numbers ix => array of form box left hand col iy => array of form box top-row iw => array of form box widths ih => array of form box heights :formhelp index(ix iy iwidth) place where iformputhelp is displayed :commandn ' ' Up to 70 characters of menu window header --------------------------------------------------- :cont sentence --------------------------------------------------- required on some :cont sentence unless :formload supplied :formshowedit iexit iexit defines exit code -------------------------------------------------- optional input commands used before :formshowedit -------------------------------------------------- :formattribute ifield key forcolor backcolor ifield = 0 used for help field key B => bold F => flashing I => italics R => reverse video U => underline N => all disabled :formattribute 7 'N' 'BRED' ' ' :defaultattribute :formbox nbox iftype forcolor backcolor clearcolor i4 i4 ch1 ch1 ch1 :formframe ifield iftype forcolor backcolor i4 i4 ch1 ch1 :formpopupmenu ifield noptn :formrangedouble ifield array(:dmin dmax) :formrangeinteger ifield index(ifmin ifmax) :formrangereal ifield array(:rmin rmax) :formputstring ifield 'string' isize isize is an optional argument that sets the length of a long string. :formputinteger ifield integer for negative # use form ifield index(-99) :formputreal ifield real 'fmt' :formputdouble ifield dvar 'fmt' :formputhelp ifield 'message' :formputformat ifield 'fmt' :formputmenu ifield choices istart :formverticalmenu ifield nseen iframe Note: placed after formputmenu. :formputbutton ifield cvalue iexitk :formputbutton 12 run 21 :formputcheckbox ifield ivalue idfield 0 or 1 ------------------------------------------------- optional input commands used after :formshowedit ------------------------------------------------- :forminfolist makes variables for the currently loaded form. nfield_1 nbox_1 ntab_1 ifx_1 ify_1 ifwid_1 iftype_1 ifiden_1 Note: If called at subroutine level data made at level > 100. :formsave 'header' 'filename' :formsave can be used to convert forms built with :formdefine to use :formload :formclearfield ifield :formgetcheckbox ifield ivalue :formgetdouble ifield real8 :formgetinteger ifield ivalue :formgetmenu ifield ioptn :formgetradiobutton ifield ivalue :formgetreal ifield real4 :formgetstring ifield string ------------------------------------------------- :final sentence ------------------------------------------------- Note: The :final sentence has no options. The :final sentence triggers the menu. Notes: The forms command has the same function as the makemenu command except for the fact that it makes matrix variables rather than macro variables and that access is provided to lower level Interacter routines. Examples: 1. Illustrate the FORMS Command Capability b34sexec matrix; /$ /$ Use this job as a template /$ call echooff; subroutine testform(ii,int4,r4,check,menu,string, menu2,r8,string2); nfields=18; ioff=3; /$ /$ type codes /$ string 1 integer 2 real 3 cycling 4 /$ push 5 double 6 vert 7 long string 8 /$ check 9 check discript 10 /$ idfield=integers(nfields); icol = index( 2 40 2 40 2 40 2 40 2 40 2 40 2 40 2 40 10 50); irow = index( 1 1 2 2 3 3 6 6 10 10 13 13 14 14 15 15 18 18)+ioff; iwidth= index(20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 10 10); itype = index( 1001 2 1001 3 10 9 1001 7 1001 1 1001 4 1001 6 1001 8 5 5); /$ /$ Defines Exit box /$ idbox =index(1); icolbox=index( 3); irowbox=index(17+ioff); iwbox =index(68); ihbox =index(3); /$ /$ Allocte a 3 by 40 character*1 array to hold character info /$ cc =c1array(3,40:); call character(hold,'Do ARIMA Model'); cc(1,)=hold; call character(hold,'Do Regression Model'); cc(2,)=hold; call character(hold,'Do Nonlinear Model'); cc(3,)=hold; call character(fmt,'(g16.8)'); call forms(:start :formdefine S idfield icol irow iwidth itype :formhelp index(2 21+ioff 68) :formdefinebox idbox icolbox irowbox iwbox ihbox :commandn 'Test Form # 1 - Shows all Options' ); call forms(:cont :formputstring 1 'This is int*4' :formputstring 3 'This is a real*4' :formputstring 5 'Check Box' :formputstring 7 'Vertical Menu Box' :formputstring 9 'String' :formputstring 10 ' ' :formputstring 11 'Cycling Menu' :formputstring 13 'Real*8 number' :formputstring 15 'Long String' :formputstring 16 ' ' 60 :formputinteger 2 index(-9) :formputcheckbox index(6 0 5) :formputhelp 2 'Enter an integer*4 here' :formrangeinteger 2 index(-99999 99999) :formputhelp 4 'Enter an real*4 here' :formputreal 4 .1 fmt :formrangereal 4 array(:-999.,999.) :formputhelp 8 'This is a vertical menu - we show 2' :formputmenu 8 cc 1 :formverticalmenu 8 2 999 :formputmenu 12 cc 1 :formputhelp 10 'Enter a short string here' :formputhelp 12 'Click to cycle' :formputdouble 14 99.9 fmt :formrangedouble 14 array(:-999.,999.) :formputhelp 14 'This is a real*8 input menu' :formputhelp 16 'This is a long string menu' /$ /$ Exit group type 5 push /$ :formputbutton 17 'Run' 21 :formputhelp 17 'Run the Menu' :formattribute 17 'N' 'byellow' ' ' :formputhelp 18 'Escape without running' :formputbutton 18 'Escape' 23 :formattribute 18 'N' 'bred' ' ' /$ :formshowedit ii /$ :forminfolist /$ /$ pull out data into b34s matrix command names /$ :formgetinteger 2 int4 :formgetreal 4 r4 :formgetcheckbox 6 check :formgetmenu 8 menu :formgetstring 10 string :formgetmenu 12 menu2 :formgetdouble 14 r8 :formgetstring 16 string2 ); call forms(:final); return; end; call testform(ii,int4,r4,check,menu,string,menu2,r8,string2); /$ forminfolist data /$ call print('nfield_1 ',nfield_1:); /$ call print('nbox_1 ',nbox1 :); /$ call tabulate(ntab_1 ifx_1 ify_1 ifwid_1 iftype_1 ifiden_1); if(ii.eq.21)then; call print('ii =',ii:); call print('int =',int4:); call print('r4 =',r4:); call print('check =',check:); call print('menu =',menu:); call print('string =',string ); call print('menu2 =',menu2:); call print('r8 =',r8:); call print('string2=',string2); endif; if(ii.eq.23)call print('Menu terminated at user request'); b34srun; ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2. Shows Tabbed Menu b34sexec matrix; call echooff; subroutine testform(ii,int4,r4,check,menu,string, menu2,r8,string2); nfields=18; /$ type codes string 1 integer 2 real 3 cycling 4 /$ push 5 double 6 vert 7 long string 8 /$ check 9 check discript 10 idfield=integers(nfields); icol =index( 2 40 2 40 2 40 2 40 2 40 2 40 2 40 2 40 10 50); irow =index( 1 1 2 2 3 3 6 6 1 1 3 3 5 5 6 6 16 16); iwidth=index(20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 10 10); itype =index( 1001 2 1001 3 10 9 1001 7 1001 1 1001 4 1001 6 1001 8 5 5); idbox =index(1 2); icolbox=index(3 3); irowbox=index(15 15); iwbox =index(68 68); ihbox =index( 3 3); cc =c1array(3,40:); call character(hold,'Do ARIMA Model'); cc(1,)=hold; call character(hold,'Do Regression Model'); cc(2,)=hold; call character(hold,'Do Nonlinear Model'); cc(3,)=hold; call character(fmt,'(g16.8)'); call forms(:start :formdefine t idfield icol irow iwidth itype :formhelp index(2 20 68) :formdefinebox idbox icolbox irowbox iwbox ihbox :formdefinetabs t array(2:'first','second') index(8,16) index(1,2) :commandn 'Test Form # 1 - Shows Tabbed form with global'); call forms(:cont :formputstring 1 'This is int*4' :formputstring 3 'This is a real*4' :formputstring 5 'Check Box' :formputstring 7 'Vertical Menu Box' :formputstring 9 'String' :formputstring 10 ' ' :formputstring 11 'Cycling Menu' :formputstring 13 'Real*8 number' :formputreal 4 .1 fmt :formputdouble 14 99.9 fmt :formrangedouble 14 array(:-999.,999.) :formrangereal 4 array(:-999.,999.) :formrangeinteger 2 index(-99999 99999) :formputstring 15 'Long String' :formputstring 16 ' ' 60 :formputbutton 17 'Run' 21 :formputbutton 18 'Escape' 23 :formattribute 17 'N' 'byellow' ' ' :formattribute 18 'N' 'bred' ' ' :formputinteger 2 index(-9) :formputcheckbox index(6 0 5) :formputhelp 2 'Enter an integer*4 here' :formputhelp 4 'Enter an real*4 here' :formputhelp 8 'This is a vertical menu - we show 2' :formputmenu 8 cc 1 :formverticalmenu 8 2 999 :formputmenu 12 cc 1 :formputhelp 10 'Enter a short string here' :formputhelp 12 'Click to cycle' :formputhelp 14 'This is a real*8 input menu' :formputhelp 16 'This is a long string menu' :formputhelp 17 'Run the Menu' :formputhelp 18 'Escape without running' :formshowedit ii /$ :forminfolist :formgetinteger 2 int4 :formgetreal 4 r4 :formgetcheckbox 6 check :formgetmenu 8 menu :formgetstring 10 string :formgetmenu 12 menu2 :formgetdouble 14 r8 :formgetstring 16 string2 ); call forms(:final); return; end; call testform(ii,int4,r4,check,menu,string, menu2,r8,string2); /$ forminfolist data /$ call print('nfield_1 ',nfield_1:); /$ call print('nbox_1 ',nbox1 :); /$ call tabulate(ntab_1 ifx_1 ify_1 ifwid_1 iftype_1 ifiden_1); if(ii.eq.21)then; call print('ii =',ii:); call print('int =',int4:); call print('r4 =',r4:); call print('check =',check:); call print('menu =',menu:); call print('string =',string ); call print('menu2 =',menu2:); call print('r8 =',r8:); call print('string2=',string2); endif; if(ii.eq.23) call print('Menu terminated at user request'); b34srun; 3. Tests Loading a Production File b34sexec matrix; call forms(:start :formload 'iighco6.ifd' S); call forms(:cont :forminfolist); call names(all); call print('# of Fields ',nfield_1:); call print('# of Boxes ',nbox_1 :); call print('# of Tabs ',ntab_1 :); call tabulate(ifx_1,ify_1,ifwid_1,iftype_1,ifiden_1); b34srun; FORPLOT Forecast Plot using GRAPHP call forplot(y,yhat,se,se2,title); This command is subject to changes in the arguments. Command has to be loaded with call load(forplot); subroutine forplot(y,yhat,se,se2,title,file); /$ /$ y => Actual Data /$ yhat => Forecast /$ se => Positive SE /$ se2 => Negative SE /$ title => Title /$ /$ ********************************************** /$ Version 18 July 2001 /$ ********************************************** Example: b34sexec matrix; y=rn(array(20:)); yhat=rn(array(4:)); error=dfloat(integers(4))/2.; se =error+yhat; se2 =yhat - error; call character(title,'Test Forecast Plot'); call load(forplot); /$ Graph using graph call graph(y :pgborder :heading 'graph command' :htitle 2. 2. :pgxscaletop 'I' :pgyscaleleft 'NT' :pgyscaleright 'I' :colors black bred ); /$ Forplot using graphp call forplot(y,yhat,se,se2,title,' '); b34srun; GARCH2P GARCH Model Estimation using 2 pass method call garch2p(data,nar,nma,coef1,se1,t1,gnar,gnma, coef2,se2,t2,res1,res2,refine); Estimate ARMA / GARCH model following Enders (1995, page 150) two pass method. Use of this subroutine requires the command call load(garch2p); See GARCH2PA for automatic two pass method. There is also an interactive version with graphics under the matrix command. GARCH2PA is in the staging2.mac file. Arguments: Data => Data nar => # of ar terms for first moment nma => # of ma terms for first moment coef1 => first moment coefficients se1 => first moment se t1 => first moment t gnar => second moment # of ar terms gnma => second moment # of ma terms coef2 => second moment coef se2 => second moment se t2 => second moment t res1 => first moment residual res2 => second moment residual refine => if NE 0 refine models Test cases: ARMA_6, GARCH2P For a reference See Enders (1995, page 150). Example: b34sexec options ginclude('gas.b34'); b34srun; /$ User is controlling model b34sexec matrix; call loaddata; call load(garch2p); nar=6; nma=0; gnar=1; gnma=0; call garch2p(gasout,nar,nma,coef1,se1,t1,gnar,gnma, coef2,se2,t2,res1,res2,2.0); call graph(res1); call graph(res2); acf1=acf(res1); call graph(acf1); acf2=acf(res2); call graph(acf2); call tabulate(acf1,acf2); b34srun; GARCH2PF GARCH Model Estimation - 2 pass method with forecasts call garch2pf(data,nar,nma,coef1,se1,t1,gnar,gnma, coef2,se2,t2,res1,res2,refine,fbase1,nf1,fbase2, nf2,obs1,f1,conf1,obs2,f2,conf2); Estimate ARMA / GARCH model following Enders (1995, page 150) two pass method. Use of this subroutine requires the command call load(garch2pf); See GARCH2PA for automatic two pass method. There is also an interactive version with graphics under the matrix command. GARCH2PA is in the staging2.mac file. Arguments: Data => Data nar => # of ar terms for first moment nma => # of ma terms for first moment coef1 => first moment coefficients se1 => first moment se t1 => first moment t gnar => second moment # of ar terms gnma => second moment # of ma terms coef2 => second moment coef se2 => second moment se t2 => second moment t res1 => first moment residual res2 => second moment residual refine => if NE 0 refine models fbase1 => Forecast base for first moment nf1 => # of first moment forecasts fbase2 => Forecast base for second moment nf2 => # of second moment forecasts obs1 => Observation for forecast of first moment f1 => forecast for first moment conf1 => confidence intervals for first moment obs2 => Observation for forecast of second moment f2 => forecast for second moment conf2 => confidence intervals for second moment Test case: GARCH2PF For a reference See Enders (1995, page 150) Example: /$ /$ User attempts AR model with 10 terms /$ b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(garch2pf); * This setting is too big but tests software ; * For a more excessive example see ARMA_6 ; nar=10; nma=0; gnar=1; gnma=0; fbase1=norows(gasout); nf1=10; fbase2=fbase1; nf2=nf1; call garch2pf(gasout,nar,nma,coef1,se1,t1,gnar,gnma,coef2, se2,t2,res1,res2,2.0,fbase1,nf1, fbase2,nf2,obs1,f1, conf1,obs2,f2,conf2); call graph(res1); call graph(res2); acf1=acf(res1); call graph(acf1); acf2=acf(res2); call graph(acf2); call tabulate(acf1,acf2); call tabulate(obs1,f1,conf1,obs2,f2,conf2); b34srun; GAMFIT Generalized Additive Model Estimation call gamfit(y x[predictor,3] z[predictor,2]{2} :options); Implements the gamfit command to provide estimation of GAM (Generalized Additive Models) following work at Stanford by Hastie and Tibshtiani. y can be continuous or 0-1. The GAMFIT command has also been implemented in "stand alone" form as b34sexec gamfit; GAMFIT implements code developed by Hastie-Tibshirani (1986, 1990) that is in the public domain. A basic references are: Hastie, T. J. and Tibshirani (1986) "Generatized Additive Models (with discussion)," Statistical Science, 1, 297-310 Hastie, T. J. and Tibshirani, R. J. (1990) "Generalized Additive Models," New York: Chapman and Hall Faraway, Julian. "Extending the Linear Model with R" 2006, New York: Chapman & Hall/CRC Hastie, T., R. Tibshirani and J. Friedman "The Elements of Statistical Learning: Data Mining, Inference, and Prediction." 2001, 2009 New York: Springer. The basic idea of a GAM model is to estimate a non parametric regression that drops the assumption of linearity. The user sets the degrees of freedom of each independent variable. A setting df=1 implies a linear model. Hastie and Tibshirani were PhD students at Stanford in the 1980's and are now on the faculty there. Variables created %res - Residuals %y - Y variable %yhat - Predicted y %yvar - Y variable name %names - Names in Model %lag - Lag %vartype - Variable type %df - DF of variable %link - Linktype of Model %dist - Error Distribution %nob - Effective number of observations. %coef - Coefficient with constant in location 1 unlike olsq which saves in the last location. The reason for this is to line up with %z %nl_p etc. %z - z score. se = %coef/%z. %nl_p - Test for nonlinearity %dof - Degrees of freedom. Defined as the sum of the degrees of the smoother for all right hand side variables. If these were restricted to be linear, then %dof = k which is the linear case. %rss - Residual sum of squares %tss - Total sum of squares %ss_rest - Restricted Sum of Squares %sigma2 - Scale Factor = %rss/(N-sum(%dof)) As degree increases %rss falls which tends to lower %rss. Since the denominator falls also, if the decrease in numerator > than the decrease in the denominator the increase in the degree of the smoothers was warrented. Otherwise the model was over fit. %spline - Spline values. Obtained when :savex has been supplied. %smoothx - Smoothed x values. Obtained when :savex has been supplied. %x - Obtained when :savex has been supplied. Note: testnl= (%ss_rest - %rss)/%sigma2; %nl_p =chisqprob(testnl,%dof); Reported R**2 =(%tss-%rss)/%tss; Note: If lags are present, then based on the minimum lag, the following is saved. This will not occur if future right hand side variables are present and :holdout is not present. %xfobs - Observation number. %xfuture - Same as %x but for out of sample data that is available. Options supported :print Show output. :info Shows iteration summary table. :noint No intercept is estimated. :punch_sur Makes a fsave file for each variable on the right having name SVAR_____n. Variables saved are: obsnum => Obs number x_var => Actual data smooth_x => Smoothed x(j) lower => s(x)-1.96*se upper => s(x)+1.96*se part_res => Partial Residual = smooth_x + res spline => Spline(i,j) Note: smooth_x = slope(j)* (x(i,j)-mean(x(i)) + spline(i,j) i=1,n j=1,k Note: smooth_x has been calculated so that E(smooth_x)=0. An OLS model of the form y = f(smooth_x1 smooth_x2) produces coef ~ 1.0 part_res = res + smooth_x and "backs out" effect of possibly nonlinear series. Note: punch_sur gives more info but no residual list. Usually this added calculation is not needed. For detail on the calculation of upper and lower see Hastie- Tibshirani (1990, page 60) lower=effect- (1.96d+00*dsqrt(sigma2*var(i,k))) upper=effect+ (1.96d+00*dsqrt(sigma2*var(i,k))) var( ) diagonal of S where variable = Sx se = dsqrt(sigma2*var(i,k)) :basedf r8 Sets degree of fit if [predictor,r8] not supplied. Default = 3. :punch_res Saved in member GAM_RES are: obsnum => obs number y => y yhat => x residual => residual Saved in files SCOEF__1 obsnum => Observatiion Number smooth_x => Smoothed x x_var => X variable lower => Lower bound on smoothed x upper => Upper bound on smoothed x :filename=' ' Sets file name if output is requested. Unit used is 44. Default gamfit.fsv :dist type Sets error distribution. Allowed values are: gauss => gaussian (This is the default) binom => binomial poiss => Poisson gamma => gamma cox => cox :link linktype Sets link function ident (default) inver logit logar cox :tol array(:r1 r2) Sets inner and outer loop convergence. Defaults are array(:.1d-8, .1d-8) :maxit index(i1,i2) Sets Maximum number of iterations for backfitting and local scoring respectively. :holdout n Sets number of observations to hold out. :sample mask Specifies a mask real*8 variable that if = 0.0 drops that observation. Note :sample cannot be used with :holdout. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better choice is the :holdout option. :savex - Saves the X matrix in %X. This is useful for TAR modeling. Also saved is the smoothed x values in %SMOOTHX and the spline values in %SPLINE. If upper and lower bounds needed, say for plots, see :gam_surf option which saves %SMOOTHX and %SPLINE in a fsave file.. call gamfit(y x[predictor,1]{1 to 6} z[factor,4] :link ident); The model specification involves specification of the type of variable and optionally a lag or lags. Unless :noint is supplied, a constant will be automatically added to the model. The model specification allows the lags to be set in the command. Both vectors and a matrix can be supplied in this release. If no [ ] is supplied, [predictor,3] is assumed. If a matrix is supplied, no lag can be supplied. The specification call gamfit(y y[predictor,1]{1} x[predictor,2]{0 to 3} z[predictor,3]{1} )$ is the same as call gamfit(y y[predictor,1]{1} x[predictor,1] x[predictor,2]{1} x[predictor,2]{2} x[predictor,2]{3} z[predictor,3]{1})$ If a 0-1 variable is on the right setting vname[factor,1] can be used. Note that [factor,i] implies that thgere are i+1 catagories of the variable. If the variable vname has > 20 catagories, use the setting vname[predictor,1]. Examples: Call gamfit(y x1 x2[predictor,3] x3[predictor,4]{1} x4[predictor,3]{1 to 6} :print); Note: while x1 is allowed x1{1} is not since [ ] is missing. The commands x=catcol(x1 x2 x3 x4); call gamfit(y x[predictor,6] :print); illustrate matrix input in the command. Discussion of variable types and how to use command. In the model specification inside [vtype, df] the first argument is a variable type key word while the second is the degrees of freedom. Variable types response, weight and censoring variables must have df=0. Response is automatically added to the left hand variable which automatically has its DF set to 0.0 For a predictor df=1 means a linear fit, while df > 1 means a nonparametric fit with the desired degrees of freedom df. (A df=0 excludes the variable and should not be used.) A factor is a categorical variable, its df must be 1 less than the number of distinct values. The :dist parameter indicates the error model which can be set as gauss, binom, poiss, gamma or cox. The :link parameter sets the link function. Valid settings are: ident, inver, logit, logar or cox. The :tol parameter specifies the convergence thresholds for the outer and inner loops of the local scoring procedure. Output contains the analysis of deviance table that includes the slope and standard error of the linear part of the fit, plus "nl-pval" a nonlinear pvalue that tests whether a function estimate is nonlinear (large p-value (GE .95) is evidence for nonlinearity). While it is hard to use qamfit to test a hypothesis due to the difficulty of interpretaion of the coefficients on the spline smoothed data, plots of the spline function against conditional residual values, may give an indication of the presence of latent nonlinearity in the model that would suggest a specific functional form to be investigated. Example: b34sexec options ginclude('b34sdata.mac') member(gam); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call echooff; call gamfit(y age[predictor,3] start_v[predictor,3] numvert[predictor,3] :link logit :dist gauss :maxit index(2000,1500) :tol array(:.1d-13,.1d-13)); b34srun; Example of a fit out of sample /; /; Problem studied by Hastie and Tibshirani page 2,10 /; b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(polyfit); call load(polyval); call load(gamfore); call echooff; iholdout=20; call olsq(lpeptide age bdeficit :print :holdout iholdout); %olscoef=%coef; %olsfore=%xfuture*%olscoef; call gamfit(lpeptide age[predictor,3] bdeficit[predictor,3] :dist gauss :print :savex :punch_sur :holdout iholdout); yhatold=%yhat; oldx=%x; newx=%xfuture; degmod=3; oldmodel=%coef; iprint=0; call gamfore(%spline,oldx,newx,degmod,%coef,%gamfore, %link,%vartype,%df,iprint); actual=lpeptide(integers(%nob+1,norows(lpeptide))); errorols=vfam(%olsfore)-vfam(actual); errorgam=vfam(%gamfore)-vfam(actual); call tabulate(%xfobs,%gamfore,%olsfore,actual, errorols,errorgam); call graph(errorols errorgam :heading 'Out of sample error' :nolabel :nocontact :pgborder); call print(' ':); call print('Sum squared out of sample Error OLS ', sumsq(errorols):); call print('Sum squared out of sample Error GAM ', sumsq(errorgam):); b34srun; Examples with plots and plots and file creation: /; /; Linear = OLS /; /; Shows possible gains of going nonlinear /; b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(gamplot); call echooff; /; calling OLS and testing against GAMFIT call olsq( cpeptide age bdeficit :print); %olsyhat=%yhat; %olsres =%res; file='gam_3.fsv'; call gamfit(cpeptide age[predictor,3] bdeficit[predictor,3] :punch_sur :punch_res :filename file :print); call gamplot(%names,%lag,file,%olsyhat,%olsres,0); b34srun; /; Example Using Gas Data with Lags /; Illustrates call gamfit options b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(gamplot); call echooff; maxlag=3; call olsq(gasout gasout{1 to maxlag} gasin{1 to maxlag} :print); %olsyhat=%yhat; %olsres =%res; file='gam_2.fsv'; call gamfit(gasout gasout[predictor,7]{1 to maxlag} gasin[predictor,8]{1 to maxlag} :print :punch_sur :punch_res :filename file ); call gamplot(%names,%lag, file,%olsyhat,%olsres,1); b34srun; Example Gausian, GAMMA, POISSON Models with different links b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec list; b34srun; b34sexec reg; model lpeptide=age bdeficit; b34srun; b34sexec matrix; call loaddata; call load(polyfit); call load(polyval); call load(gamfore); call echooff; /; call print(polyfit,polyval,gamfore); call olsq(lpeptide age bdeficit :print); program testfore; call gamfit(lpeptide age[predictor,3] bdeficit[predictor,3] :dist argument(distk) :link argument(linkk) :print :savex :punch_sur); /; validate forecasting capability yhatold=%yhat; oldx=%x; newx=oldx; oldmodel=%coef; degmod=6; call gamfore(%spline,oldx,newx,degmod,%coef,fore, %link,%vartype,%df,iprint); if(ilist.ne.0)then; call print( 'Test forecast by looking at in simple forecast & yhat':); error=yhatold-fore; call tabulate(yhatold,fore,error); endif; return; end; /; =8 => high forecasting accuracy degmod=8; iprint=0; ilist=1; distk='gauss'; linkk='ident'; call testfore; linkk='inver'; call testfore; linkk='logar'; call testfore; /; +++++++++++++++++++++++++++++++++++++++++++++++++++++++ distk='poiss'; linkk='ident'; call testfore; linkk='inver'; call testfore; linkk='logar'; call testfore; /; +++++++++++++++++++++++++++++++++++++++++++++++++++++++ distk='gamma'; linkk='ident'; call testfore; linkk='inver'; call testfore; linkk='logar'; call testfore; b34srun; Example: Saving data from gamfit to be moved to another matrix step of from unit to the PC /; Illustrates saving data and plotting on a PC in a later step b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(gamplot); call echooff; /; calling OLS and testing against GAMFIT call olsq( cpeptide age bdeficit :print); %olsyhat=%yhat; %olsres =%res; file='gam_3.fsv'; call gamfit(cpeptide age[predictor,3] bdeficit[predictor,3] :punch_sur :punch_res :filename file :print); %gamyhat=%yhat; %gamres =%res; call checkpoint(:var %names %lag file %olsyhat %olsres %gamyhat %gamres :file 'gamsave.por'); b34srun; /; From unix we move gamsave.por and gam_3.fsv b34sexec matrix; call load(gamplot); call restore(:file 'gamsave.por'); file='gam_3.fsv'; call gamplot(%names,%lag,file,%olsyhat,%olsres,0); b34srun; Example: Test job of ACE vs GAM on 0-1 variable b34sexec options ginclude('b34sdata.mac') member(gam); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call echooff; call load(ace_ols); call acefit(y[cat] age[order] start_v[order ] numvert[order] :print); call ace_ols; call gamfit(y age[predictor,3] start_v[predictor,3] numvert[predictor,3] :link logit :dist gauss :maxit index(2000,1500) :tol array(:.1d-13,.1d-13) :print); call print(%tss,%rss,%sigma2); call tabulate(%coef,%z,%nl_p,%ss_rest); /; Here age restricted call gamfit(y age[predictor,1] start_v[predictor,3] numvert[predictor,3] :link logit :dist gauss :maxit index(2000,1500) :tol array(:.1d-13,.1d-13) :print); call print(%tss,%rss,%sigma2); call tabulate(%coef,%z,%nl_p,%ss_rest); call names; b34srun; GAMFORE Forecast a GAM Model call gamfore(sp,oldx,newx,degmod,oldmodel,fore,link, vartype,dfmod,iprint); Forecast a gam model sp => spline set as %spline if :savex supplied to gamfit oldx => old x vector newx => new x vector degmod => degree of polynomial fit of spline; Should be set GE 3 oldmodel => set as %coef from call gamfit( ); step fore => forecast link => Set as %link from gamfit vartype => Set as %vartype from gamfit dfmod => Set as %df from gamfit iprint => Print internal spline equations if =1 Note: requires polyfit and polyval be loaded arguments for polyfit and polyval call polyfit(x,y,n,coef,printout)$ call polyval(coef,xin,yhat)$ logic: Obtain coef of mapping of X to spline. From XB we note that: yhat + sum(spline) = XB yhat = XB - sum(spline) Analysis assumes expected spline coef = 1.0 in large samples Example of a fit out of sample /; /; Problem studied by Hastie and Tibshirani page 2,10 /; b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(polyfit); call load(polyval); call load(gamfore); call echooff; iholdout=20; call olsq(lpeptide age bdeficit :print :holdout iholdout); %olscoef=%coef; %olsfore=%xfuture*%olscoef; call gamfit(lpeptide age[predictor,3] bdeficit[predictor,3] :dist gauss :print :savex :punch_sur :holdout iholdout); yhatold=%yhat; oldx=%x; newx=%xfuture; degmod=3; oldmodel=%coef; iprint=0; call gamfore(%spline,oldx,newx,degmod,%coef,%gamfore, %link,%vartype,%df,iprint); actual=lpeptide(integers(%nob+1,norows(lpeptide))); errorols=vfam(%olsfore)-vfam(actual); errorgam=vfam(%gamfore)-vfam(actual); call tabulate(%xfobs,%gamfore,%olsfore,actual, errorols,errorgam); call graph(errorols errorgam :heading 'Out of sample error' :nolabel :nocontact :pgborder); call print(' ':); call print('Sum squared out of sample Error OLS ', sumsq(errorols):); call print('Sum squared out of sample Error GAM ', sumsq(errorgam):); b34srun; Example on an in-sample validation of forecasting code: b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec list; b34srun; b34sexec reg; model lpeptide=age bdeficit; b34srun; b34sexec matrix; call loaddata; call load(polyfit); call load(polyval); call load(gamfore); call echooff; call print(polyfit,polyval,gamfore); call olsq(lpeptide age bdeficit :print); call gamfit(lpeptide age[predictor,3] bdeficit[predictor,3] :dist gauss :print :savex :punch_sur); /; validate forecasting capability yhatold=%yhat; oldx=%x; newx=oldx; degmod=3; oldmodel=%coef; iprint=0; degmod=6; call gamfore(%spline,oldx,newx,degmod,%coef, fore,%link,%vartype,%df,iprint); call print('Shows error of the forecast':); error=yhatold-fore; call tabulate(yhatold,fore,error); call graph(yhatold fore :heading 'Forecasting inside sample' :nolabel); call graph(Error :heading 'Forecasting generatioon error'); b34srun; GAMPLOT Display GAMFIT Results call gamplot(names1,lag1,file,olsyhat,olsres,iprint); Graph data from GAMFIT *.fsv file Assumes :punch_sur and :punch_res was supplied and user Runs OLS prior to gamfit command %names,%lag, are automatically created by gamfit command %olsyhat,%olsres are the old yhat and the oldres from the olsq step iprint is coded =0 => display leverage graph =1 => save leverage graphs in a file with same name as fsv file =2 => display and save leverage graphs =10 => Same as 0 except raw and smoothed series shown in addition =11 => Same as 1 except raw and smoothed series shown in addition =12 => Same as 2 except raw and smoothed series shown in addition Example: b34sexec options ginclude('b34sdata.mac') member(gam_3); b34srun; b34sexec options noheader; b34srun; b34sexec matrix; call loaddata; call load(gamplot); call echooff; /; calling OLS and testing against GAMFIT call olsq( cpeptide age bdeficit :print); %olsyhat=%yhat; %olsres =%res; file='gam_3.fsv'; call gamfit(cpeptide age[predictor,3] bdeficit[predictor,3] :punch_sur :punch_res :filename file :print); %gamyhat=%yhat; %gamres =%res; call gamplot(%names,%lag,file,%olsyhat,%olsres,0); b34srun; GARCH Calculate function for a ARCH/GARCH model. call garch(res,arch,y,func,maxlag,nbad :options); The GARCH subroutine supports a general way to setup a GARCH/ARCH/GARCH-M model and avoid the overhead of recursive calls. The GARCH command works with one series although more than one series can be on the right. The GARCH subroutine calculates the function which is then maximized with CMAXF2 or the in more complex cases with the nonlinear programing with nonlinear constraints command NLPMIN1. The advantage of GARCH over GARCHEST is that constraints can be placed on parameters and constrained maximizer routines other than CMAXF2 can be used. In addition the parameters can optionally be observed as they change. For most GARCH applications, GARCHEST should be used. GARCHEST has been enhanced for alternative models. Since the solution depends on only func, if an alternative model not built into GARCH is needed, the func can be recalculated in the user's routine. GARCH modeling in RATS often has a problem with "useable" observations that arises because during the iteration phase in the second moment equation the value goes LE 0 causing problems with the LOG and the division. If GARCH is used with the CMAXF2 command it is possible to restrict the parameters of the second moment equation such that this does not occur. Sample jobs GARCH3,.., GARCH7 illustrate the use of the GARCH subroutine. The key sections of these jobs are listed below. In the complete jobs, RATS commands are supplied so as to benchmark the results. The job GARCHEST_3 shows GARCHEST on the McCullough-Renfro benchmark. GARCH_6 shows the same test case using the GARCH subroutine and calling CMAX2 directly. This tests case illustrates how the initial values in res1 and res2 make a difference. The b34s GARCH subroutine is slower than Rats, but provides complete instrumentation of the solution process and will not give the "useable" observations message. The GARCHEST command is 4-5 times faster than the GARCH/CMAXF2 combination and should be used for most cases. If speed is NOT an issue and a custom model is estimated, then the model should be hand coded in a matrix command subroutine. This will not be fast. Required GARCH Subroutine arguments res - first moment residual arch - second moment residual y - first moment variable func - function maxlag - maxlag of model for purposes of ML sum. nbad - number of bad datapoints If res1 and res2 are allocated prior to the call to GARCH, the initial values placed in these series are used. If GARCH allocates res1 and res2, all values are set to 0.0. Options supported :AR arparm arorder - AR parameters & orders :MA maparm maorder - MA parameters & orders :GAR garparm garorder - GAR parameters & orders :GMA gmaparm gmaorder - GMA parameters & orders :MU muparm muorder - Mu parameters and order :NOSQRT if present does not take squrt for garch-M models :CONSTANT cparm - Constant :XVAR xmatrix xparm xorder maxlagvec - X matrix parms. orders lags xmatrix - data matrix for inputs xparm - parameter names xorder - vector of orders for inputs maxlagvec - number of parameters for each input :FORECAST Produces %F_M1_M2 a two element array containing first and second moment forecast for last observation. Model estimated max -.5 * (dlog(res2) + ((res1**2)/res2) ) where res1(t)=y(t)-cparm(1)-arparm(1)*y(t-arorder(1))-... -maparm*res(t-maorder(1))-... -muparm(1)*dsqrt(arch(t-muorder(1)))-... -xparm(1)*x1(t-xorder(1))-... arch(t)=cparm(2)+gmaparm(1)*(res(t-gmaorder(1))**2) + garparm(1)* arch(garaorder(1) + ... Note: If overflows occur the parameters of the model may have to be restricted in such a way that they do not get near 0.0 during the solution iterations. Since the order in which the equations are solved is res1 and res2, if muorder(1)=0, then the system will have to be coded with DO loops. Examples of alternative coding are contained in ENDERS2 and ENDERS2B jobs in b34stest.mac. Sample Jobs GARCH3 Joint GARCH(0,1) Estimation /$ /$ Joint GARCH(0,1) Estimation using GARCH subroutine /$ b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; call loaddata; count=0.0; j=norows(gasout); arch = array(j:)+1.; res = array(j:); archlog= array(j:); call echooff; program test; /$ Using built in garch subroutine func=0.0; count=count+1.0; call garch(res,arch,gasout,func,2,n :ar array(:b1,b2) idint(array(:1 2)) :gma array(:a1) idint(array(:1) ) :constant array(:b0 a0) ); call outstring(4,3,'F count a0 a1 b0 b1 b2'); call outdouble(34,3,func); call outdouble(54,3,count); call outdouble(4, 4, a0); call outdouble(24,4, a1); call outdouble(44,4, b0); call outdouble( 4,5, b1); call outdouble(24,5, b2); return; end; call print(test); /$ tests a1=.05; /$ /$ Get starting values /$ call olsq(gasout gasout{1 to 2} :print); call print(%coef); call cmaxf2(func :name test :parms b0 b1 b2 a0 a1 :ivalue array(:%coef(3),%coef(1),%coef(2),%resvar,a1) :maxit 300 :gradtol .1e-4 :lower array(:-.1d+30,-.1d+30,-.1d+30,0.0,0.0) :upper array(: .1d+30, .1d+30, .1d+30,.1d+30,.1d+30) :print); call print('Number out of function ',n); call print(sumsq(res)); call tabulate(res,arch); b34srun; ******************************************************** GARCH4 Joint GARCH(1,1) Estimation b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; call loaddata; j=norows(gasout); count=0.0; arch = array(j:); res = array(j:); archlog= array(j:); call echooff; program test; /$ Using built in garch subroutine func=0.0; count=count+1.0; call garch(res,arch,gasout,func,2,n :ar array(:b1,b2) idint(array(:1 2)) :gma array(:a1) idint(array(:1) ) :gar array(:a2) idint(array(:1) ) :constant array(:b0 a0) ); call outstring(4,3,'Function'); call outdouble(24,3,func); call outdouble(64,3,count); call outdouble(4, 4, a0); call outdouble(24,4, a1); call outdouble(44,4, a2); call outdouble(64,4, b0); call outdouble( 4,5, b1); call outdouble(24,5, b2); return; end; call print(test); /$ tests a1=.05; a2=0.0; /$ /$ Get starting values /$ call olsq(gasout gasout{1 to 2} :print); call print(%coef); /$ call cmaxf2(func :name test :parms b0 b1 b2 a0 a1 a2 :maxit 200 :maxsteps 10. :ivalue array(:%coef(3),%coef(1),%coef(2),%resvar,a1,a2) :maxit 300 :gradtol .1e-4 :lower array(:-.1d+30,-.1d+30,-.1d+30,.1d-16,.1d-16, -.1d-16) :upper array(: .1d+30, .1d+30, .1d+30,.1d+30,.1d+30, .1d+30) :print); b34srun; ************************************************************ GARCH5 Transfer Function Model with GARCH. Rats used to test b34sexec options ginclude('gas.b34'); b34srun; /$ /$ Estimate a GARCH Transfer function. /$ RATS used to validate results. /$ b34sexec matrix ; call loaddata; j=norows(gasout); count=0.0; arch = array(j:); res = array(j:); archlog= array(j:); call echooff; program test; /$ Using built in garch subroutine to estimate a GARCH /$ Transfer function func=0.0; count=count+1.0; call garch(res,arch,gasout,func,3,n :ar array(:b1,b2) idint(array(:1 2)) :gma array(:a1) idint(array(:1) ) :gar array(:a2) idint(array(:1) ) :xvar gasin array(:gin1 gin2) idint(array(:1 3)) idint(array(:2)) :constant array(:b0 a0) ); call outstring(4,3,'Function'); call outdouble(24,3,func); call outdouble(64,3,count); call outdouble(4, 4, a0); call outdouble(24,4, a1); call outdouble(44,4, a2); call outdouble(64,4, b0); call outdouble( 4,5, b1); call outdouble(24,5, b2); call outdouble( 4,6, gin1); call outdouble(24,6, gin2); return; end; call print(test); /$ tests a1=.01; a2=.01; /$ /$ Get starting values /$ call olsq(gasout gasout{1 to 2} gasin{1} gasin{3} :print); call print(%coef); /$ call cmaxf2(func :name test :parms b0 b1 b2 gin1 gin2 a0 a1 a2 :maxit 200 :maxsteps 4. :ivalue array(:%coef(5),%coef(1),%coef(2),%coef(3), %coef(4),%resvar,a1,a2) :gradtol .1e-4 :lower array(:-.1d+30,-.1d+30,-.1d+30,-.1d+30,-.1d+30, .1d-16,.1d-16,-.1d-16) :upper array(: .1d+30, .1d+30, .1d+30, .1d+30, .1d+30, .1d+30,.1d+30, .1d+30) :print); call print(sumsq(res)); call tabulate(res,arch); b34srun; /$ /$ BHHH method used..residuals set to 0 for beginning obs /$ /$ User must replace GASOUT with user series name /$ b34sexec options open('rats.dat') unit(28) disp=unknown$ b34srun$ b34sexec options open('rats.in') unit(29) disp=unknown$ b34srun$ b34sexec options clean(28)$ b34srun$ b34sexec options clean(29)$ b34srun$ b34sexec pgmcall$ rats passasts pcomments('* ', '* Data passed from B34S system to RATS', '* ') $ pgmcards$ * set seriesn = gasout compute iter = 100,isiter=100 * * garch(1,1) * smpl(series=seriesn) set u11 = 0.0 set v11 = 0.0 nonlin b0 b1 b2 gin1 gin2 a0 a1 beta1 frml regresid = seriesn-b0-b1*seriesn{1}-b2*seriesn{2} $ -gin1*gasin{1}-gin2*gasin{3} frml garchvar = a0+a1*u11{1}**2 + $ beta1 * %if(v11{1}>1.e+100,%na,v11{1}) frml garchlogl = v11(t)=garchvar(t),u11(t)=regresid(t),$ -.5*(log(v11)+u11**2/v11) linreg seriesn # constant seriesn{1} seriesn{2} gasin{1} gasin{3} compute b0=%beta(1),b1=%beta(2),b2=%beta(3),a0=%seesq, $ a1=.05 compute beta1=0.0 compute gin1=%beta(4) compute gin2=%beta(5) nlpar(subiterations=isiter) * maximize(method=simplex,recursive,iterations=iter) $ garchlogl 4 * maximize(method=bhhh,recursive,iterations=iter) $ garchlogl 4 * print * * u11 v11 smpl(series=u11) statistics u11 set rssg11 = u11(t)*u11(t) statistics rssg11 smpl(series=rssg11) compute sumsqu11 = %sum(rssg11) display 'sum of squares of u11 for garch' sumsqu11 b34sreturn$ b34srun$ b34sexec options close(28)$ b34srun$ b34sexec options close(29)$ b34srun$ b34sexec options dodos('rats386 rats.in rats.out') dounix('rats rats.in rats.out')$ b34srun$ b34sexec options npageout writeout('output from rats',' ',' ') copyfout('rats.out') dodos('erase rats.in','erase rats.out', 'erase rats.dat') dounix('rm rats.in','rm rats.out', 'rm rats.dat')$ b34srun$ ************************************************************* GARCH6 Illustrates various test problems b34sexec options ginclude('b34sdata.mac') macro(bg_test1); b34srun; /$ /$ Set dorats=1 to run RATS on the test problem /$ /$ Problem discussed in /$ "Benchmarks and Software Standards: A Case study of /$ GARCH procedures" McCullouch & Renfro /$ Journal of Economic and Social Measurement 25 (1998) /$ 59-71 /$ /$ Has Do loop and GARCH implementation /$ Do loop runs very very slowly /$ b34sexec matrix ; call loaddata; count=0.0; j=norows(returns); arch = array(j:); res = array(j:); archlog= array(j:); * one and pfive make code run faster; one=1; pfive=.5; smu=mean(returns); svar=variance(returns-smu); /$ Set starting value for h(1) if ne 0.0 /$ arch= arch+1.; /$ arch= arch+ (sumsq(returns-smu)/dfloat(j)); call echooff; program test; func=0.0; count=count+1.0; /$ Uncomment do loop and comment call garch to switch /$ mode of running /$ res=returns-mu; /$ do i=2,j; /$ arch(i)=a0+a1*(res(i-one)*res(i-one))+b1*arch(i-one); /$ func=func-(pfive*mlsum(arch(i)))- /$ (pfive*((res(i)*res(i))/arch(i))); /$ enddo; /$ /$ adjusting h(1) /$ /$ if(count.gt.1.)then; /$ arch(1)=(sumsq(res)-(res(1)*res(1)))/dfloat(j-1); /$ endif; /$ /$ Using built in garch subroutine results in faster code /$ res=returns-mu; call garch(res,arch,returns,func,1,n :gar array(:b1) idint(array(:1)) :gma array(:a1) idint(array(:1)) :constant array(:mu a0) ); call outstring(4,3,'F count mu a0 a1 b1'); call outdouble(34,3,func); call outdouble(54,3,count); call outdouble(4, 4, mu); call outdouble(24,4, a0); call outdouble(44,4, a1); call outdouble( 4,5, b1); * call print(func mu a0 a1 b1); return; end; call print(test); /$ /$ tests starting values /$ call cmaxf2(func :name test :parms mu a0 a1 b1 /$ These are benchmark starting values. /$ :ivalue array(:-.016427, .221130, .35,.50) :ivalue array(:smu, svar .01 .5) :maxit 9000 /$ :gradtol .1d-07 /$ :steptol .1d-12 :lower array(:-10., .1d-2, .1d-2, .1d-2) :upper array(: 10. 10. 10. 10. ) :print); call print(sumsq(res)); * call tabulate(res,arch); * Two pass method ; * fixedet=(returns-mean(returns))*(returns-mean(returns)); * call arma(fixedet :maxit 2000 :relerr 0.0 :nar 1 :nma 1 :print); b34srun; Example using DO loop. See GARCH_DO1 in matrix.mac b34sexec options ginclude('gas.b34'); b34srun; /$ This problem runs slow but is most general case. /$ Using DO loops can express any model. /$ Can use CMAXF2 if limit problems. b34sexec matrix ; call loaddata; /$ subset if j reduced j=norows(gasout); call print('# cases used was ',j); count=0.0; arch = array(j:); res = array(j:); archlog= array(j:); call echooff; program test; /$ Using do loop func=0.0; do ii=i,j; res(ii) =gasout(ii) - (b0 +(b1*gasout(ii-1))+ (b2*gasout(ii-2))); arch(ii) =a0 + (a1*(res(ii-1)*res(ii-1))) + a2*dmin1(dabs(arch(ii-1)),.1e+70); func=func+((-.5)*(dlog(dmax1(dabs(arch(ii)), .1e-10))+ ( (res(ii)*res(ii)) /arch(ii)))) ; * call outdouble(3,1,dfloat(ii)); * call outdouble(43,1,func); enddo; count=count+1.0; call outstring(4,3,'Function'); call outdouble(24,3,func); call outdouble(64,3,count); call outdouble(4, 4, a0); call outdouble(24,4, a1); call outdouble(44,4, a2); call outdouble(64,4, b0); call outdouble( 4,5, b1); call outdouble(24,5, b2); return; end; call print(test); i=3; /$ initial values that were set b0=2.; b1=1.7; b2=-.7; a0=.04; a1=.2; a2=.5; /$ /$ call test; /$ call stop; call maxf2(func :name test :parms b0 b1 b2 a0 a1 a2 :maxit 200 :ivalue array(:b0, b1, b2, a0, a1, a2) :print); /$ Alternative setup * call cmaxf2(func :name test :parms b0 b1 b2 a0 a1 a2 :maxit 2000 :maxfun 2000 :maxg 2000 :ivalue array(:b0, b1, b2, a0, a1, a2) :lower array(:-.1d+30,-.1d+30,-.1d+30, .1d-16, .1d-16, .1d-16) :upper array(: .1d+30, .1d+30, .1d+30, .1d+30, .1d+30, .1d+30) :print); b34srun; *********************************************************** GARCH_7 IGARCH(1,1) using NLPMIN1 - see test case GARCH_7 /$ IGARCH(1,1) using NLPMIN1 - showsgeneral case b34sexec options ginclude('b34sdata.mac') member(garchdat); b34srun; b34sexec matrix ; call loaddata; y=sp500; vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); call print('mean y ',mean(y):); call print('vstart ',vstart :); program test; call garch(res,arch,y,func,1,nbad :gar array(:gar) idint(array(:1)) :gma array(:gma) idint(array(:1)) :constant array(:a0 b0) ); if(%active(1)) g(1)=gar+gma-1.; func=(-1.)*func; return; end; call print(test); call echooff; call NLPMIN1(func g :name test :parms gar gma a0 b0 :ivalue array(:.5,.5,mean(y),vstart) :nconst 1 0 :lower array(: 1.d-6, 1.d-6, 1.d-6, 1.d-6) :upper array(: 1.d+2, 1.d+2, 1.d+2, 1.d+2) :print :maxit 100 :iprint final); b34srun; GARCHEST Estimate a ARCH/GARCH model. call garchest(res1,res2,y,func,maxlag,nbad :options); The GARCHEST subroutine provides a simple way to setup and estimate GARCH/ARCH/GARCH-M class models and avoid the overhead of both recursive calls and calling a user program. The GARCHEST command supports univariate and multiple input transfer function models and provides the fastest and simplest way to setup and run these complex models. The GARCH command and the call to CMAXF2 should be used for more complicated models or models where the calculation progress has to be monitored. The GARCHEST command will be faster than the GARCH/CMAX2 commands. The GARCHEST command provides added capability for alternative models that are not in the GARCH command which is really just a function that speeds the recursive calculation. If there is demand, the GARCH command can be extended. For complete model flexibility, use a DO loop implementation. Due to the recursive nature of the GARCH/ARCH class of models, vectorizing the calculation is not possible. The speed will be substantially less!! For complex nonlinear constraints use the NLPMIN1 command, which solves nonlinear programming models with nonlinear constraints. The GARCHEST command automatically calls the CMAXF2 routine to maximize the function. The GARCHEST arguments are very simular to GARCH and the options simular to ARMA to facilitate movement back and forth. The sample jobs GARCHEST and GARCHEST_2 illustrate the use of the GARCHEST command. The key sections of these jobs are listed below. In the complete jobs, RATS commands are supplied so as to benchmark the results and to show clearly what is being calculated. The job garchest_3 shows GARCHEST on the McCullough-Renfro benchmark. GARCH_6 shows the same test case using the GARCH subroutine and calling CMAX2 directly. This tests case illustrates how the initial values in res1 and res2 make a difference. GARCHEST Subroutine arguments res1 - first moment residual res2 - second moment residual y - first moment variable func - function maxlag - maxlag of model for purposes of ML sum nbad - number of bad datapoints If res1 and res2 are allocated prior to the call to GARCHEST, the initial values placed in these series are used. If GARCHEST allocates res1 and res2, all values are set to 0.0. Options supported :nar n - Sets n as the max AR order provided all terms up to n are to be estimated. In this case the keyword :arorder is not needed. :arorder ivec - Sets AR terms to be estimated for restricted model. :nar is not set in this case. :arparms rvec - Sets initial AR parameter values. Initial values usually set by using OLS coefficients. :nma m - Set for max MA order provided all terms up to m are to be estimated. In this case :maorder is not needed. :maorder ivec - Sets MA terms to be estimated for restricted models. :nma is not set in this case. :maparms rvec - Sets initial MA parameter values. :ngar n - Sets n as the max GAR order provided all terms up to n are to be estimated. In this case the keyword :garorder is not needed. :garorder ivec - Sets GAR terms to be estimated for restricted model. :ngar is not set in this case. :garparms rvec - Sets initial GAR parameter values. Usually not required. :ngma m - Set max GMA order provided all terms up to m are to be estimated. In this case :gmaorder is not needed. :gmaorder ivec - Sets GMA terms to be estimated for restricted models. :ngma is not set in this case. :gmaparms rvec - Sets initial GMA parameter values. Usually not required. :nmu m - Set for max MU order provided all terms up to m are to be estimated. In this case :muorder is not needed. :nmu needs to be set for GARCH-M models or :muorder needs to be set. :muorder ivec - Sets order of MU terms to be estimated for restricted models. :nmu is not set in this case. :muparms rvec - Sets initial MU parameter values. Usually not required. :noconst1 - No first moment constant. :noconst2 - No second moment constant. :noconst3 - No constant for etgarch. :cparms rvec - Pass 2 element initial constant values. Two elements passed even if :noconst1 or :noconst2 are set. If :etgarch is set, pass three element vector. :print - Print results of estimation. :lower rvec - Vector of lower values for parameters. Usually not needed. First moment parms have range -1.d+32 - 1.d+32. Second moment parameters are restricted to be > 0.0. :upper rvec - Vector of upper values for parameters Usually not needed. First moment parms have range -1.d+32 - 1.d+32. Second moment parameters are restricted to be > 0.0. :xvar xmatrix xparm xorder maxlagvec - xmatrix - data matrix for inputs - xparm - initial values. - xorder - vector of input orders - maxlagvec - number of parameters for each input Note that initial values must be supplied. :xscale vec - Vector of n elements to scale coef vector. Default = 1.0 :fscale real - Functional scaling. Default = 1.0. :ngood int - Sets number of good digits in the function. Default = 15. :maxit int - Maximum number of iterations. Default = 400. :maxfun int - Maximum number of function evaluations. Default = 400 :maxg int - Maximum number of gradiant evaluations. Default = 400 :gradtol real - Scaled gradiant tolerance. Default = eps**(1/3). :steptol real - Scaled step tolerance. Default = eps**(2/3). :rftol real - Relative functional tolerance. Default = max(1.0d-20,eps**(2/3)). :aftol real - Absolute functional tolerance. Default = max(1.0d-20,eps**(2/3)). :fctol real - False convergence tolerance. Default = 100.*eps. :maxsteps real - Maximum allowable step size. Default = (1000*max(tol1,tol2)) where for i=1,n tol1= sqrt(sum of (xscale(i)*ivalue(i))**2 tol2 = 2-norm of XSCALE :simplex - Use simplex (db2pol) to obtain better starting values. :print2 - Prints simplex results. Unless print2 is set, max iteration limit message will not be given. :print3 - Produces limited output for use under SCA Work Bench :ftol real - Relative functional tolerance for simplex. Default = max(1.0d-20,eps**(2/3)). :maxit2 int - Maximum number of iterations for simplex. Default = 100. :stop - Makes exceeding maxit iterations a fatal error that will terminate further processing of the matrix job and stop b34s. This option is usually not used and instead the user inspects the log. The purpose of this option is to kill a long batch job if there was an iteration limit exceeded early in a long job. Advanced Modeling options. There is a wide variaty of advances GARCH type models and various forms. Only a few are "hard wired" here. For exact detail on the format used, see the examples listed below. To estimate models not "hard coded," use the garch subroutine or directly code the models. This will be substantially slower although any model can be estimated as long as it can be coded in a way that B34S can parse. :fattail vd - Modifies the GARCH likelihood function for fattails. The parameter vd is calculated. If vd is NOT supplied, it is set to dfloat(n) as an initial guess for start at nornality. If the dataset has many observations, then this might not be an ideal setting since the objective function will not change much for changes in VD. Tsay (2002) has suggested starting at 5.0. This may be too small. It is suggested that for the final model estimated a number of starting value be tried. :shape vd - Sets shape. Here the fat tail distribution is set to value vd, not estimated. This option is used for :fattail :dist=0, :dist=1 and :dist=2 only. :dist i - i=0 => Rats Formula. i=1 => Tsay formula Tsay (2002) 3.8 page 89 i=2 => GED Generalized Error See Hamilton (1994, 668) i=3 => Cauchy Distribution :nosqrt - Does not take the sqrt of the second moment when estimating ARCH-M and GARCH-M models. :igarch - Estimates a GARCH(1,1) model with the constraint that gar(1)+gma(1)=1.0. If this option is used, only :ngar is supplied. If more than a IGARCH(1,1) is estimated, the DO loop or GARCH setup must be used and NLPMIN1 should be used to set the appropriate constraint. Since as a default there is no positive upper bound, using the :igarch option might cause problems if gar(1) > 1.0 since this implies gma(1) < 0. Hence when using the :igarch option it is suggested that the :upper option be used to impose an upper bound on gar(1) of 1.0. See test job GARCH_7. :egarch vd - Estimates the Nelson (1991) model where the log of the variance is modeled. The parameter vd is calculated. If vd is NOT supplied, it is set to .01 as an initial guess. The exact form is shown in Rats User's guide (2000) Version 5 page 352. For details see below. :egarch2 vd - Estimates modified form of the S-Plus EGARCH model. This form can be translated into the more traditional form. Both :egarch and :egarch2 allow parameters to go < 0.0. Hence in estimation the lower bounds may have to be adjusted with the :lower option. :gjr gma2 - The Glosten et. al (1993) tarch specification for the gma terms breaks the effect of the squared residuals in the second moment equation into two components: one for res1(i) > 0 and one for res1(i) < 0. Initial values can be optionally set after :gjr. The :gjr option requires that gma parameters be in the model. The exact form is shown in Rats User's guide (2000) Version 5 page 352. For details see below. The GJR (tarch) is a special case of teh etgarch model in that only gma terms are impacted :tgarch gar2 - The tgarch specification for the gar terms breaks the effect of res2 in the second moment equation into two components: one for res1(i) > 0 and one for res1(i) < 0. Initial values can be optionally set after :tgarch. The :tgarch option requires that gar parameters be in the model. For an example see Tsay (2002) page 133 eq. (4.12). The :tgarch mode is a special case of the :etgarch model in that only gar terms are impacted. :tgarch2 parms - The tgarch2 specification breaks up the gma into two parts. The number of parms are ngma. Here likelihood function uses second equation squared. This model "looks" like a GJR model escept for the LF used. Basic reference "Threshold Arch Models and Asymmetries in Volatility," R. Rabemananjara & J. M. Zakoian Journal of Applied Econometrics, Vol. 8, 31-49 (1993). Also mentioned in Tsay (2002) page 133 as as alternative threshold volatility model. :tgarch3 - Same as :tgarch2 except parameter constrained to be the same. Likelihood of :tgarch2 used. :etgarch parms - Estimate the etgarch model. parms optionally sets the gar, gma and constant initial values. :noconst3 can be used to turn off etgarch constant. If present, :noconst3 must be placed before :etgarch. The etgarch specification in essence is a combination of the GJR (TARCH) and TGARCH model. This form is shown in Tsay (2002) page 168 and discussed on pages 129-133. General notes: The GARCH/ARCH class of models is wide and changing. A general purpose command such as GARCHEST cannot ever hope to be able to model all possible cases. If a desired case is not modeled, the DO loop approach, while slow, can be used. The SOLVE/FORMULA approach can also be used and should probably be tried first. These approaches may run into temp variable limits. For further detail on how to avoid these problems by managing the workspace see help documents for COMPRESS and SOLVEFREE commands. Use of these commands allows the temp variables to be cleaned while the command is running. The jobs SOLVEFREE1 and SOLVEFREE2 in matrix.mac show these alternative approaches to model estimation. Detail on GARCH modeling in finance is contained in "The Econometrics of Financial Markets" by Campbell-Lo- MacKinley. Princeton 1997. See especially chapter 12. The (2002) book by Ruey Tsay contains an excellent discussion of the modern approaches. Enders (2004) is also an excellent reference on Time Series modeling. In addition to the test cases in matrix.mac, testgar.mac contains a number of jobs that should be run. Many of these jobs contain Rats implementations that are useful to study. Variables created if options are selected: %coef - Estimated coefficients in order ar, ma, gar, gma, mu vd const1 const2 %se - Coefficient Standard Errors %t - Coefficient t scores %cname - Coefficient name %corder - Coefficient order %hessian - Hessian %resobs - Observation # of residual. First MAXLAG set to missing. First MAXLAG obs of RES1 and RES2 set to missing. %g - If EGARCH model where g(t) =dabs(res1(t)/dsqrt(res2(t))) - dqsrt(2.0d+00/pi()) - vd*res1(t)/dsqrt(res2(t)) %func - Final Functional Value %nparm - # of parameters %n_goodd - # of good digits in function %n_iter - # of iterations %n_func - # of function evaluations %n_grad - # of gradiant evaluations %sg_tol - # Scaled Gradient Tolerance %ss_tol - # Scaled Step Tolerance %rf_tol - Relative Function Tolerance %fc_tol - False Convergence Tolerance %max_ss - Maximum allowable step size %s_trust - Size of Initial Trust region %nt_drop - # of terms dropped in ML %h_cond - 1/ Condition of Hessian Matrix The following automatic variables are useful for garchf subroutine that is used to make forecasts. %nar - # AR parameters %nma - # MA parameters %ngar - # GAR parameters %ngma - # GMA parameters %nmu - # MU parameters %con - # constants. 2= both, -1 = first only, 1 = second only Model estimated: ++++++++++++++++++++++++++++++++++++++++++++++++++++++ Likelihood Function in the default case: ++++++++++++++++++++++++++++++++++++++++++++++++++++++ max -.5 * (dlog(res2) + ((res1**2)/res2) ) ****************************************************** if :fattail is set the function maximized is max tden((res1/dsqrt(res2)),df) - .5*dlog(res2) where tden(x,df) is the density of the t distrution for x at degrees of freedom df. ****************************************************** If :tgarch2 or :tgarch3 are estimated the function maximized is max -.5(dlog(res2**2)+((res1**2)/(res2**2)) ++++++++++++++++++++++++++++++++++++++++++++++++++++++ Default First Moment equation: ++++++++++++++++++++++++++++++++++++++++++++++++++++++ res1(t)=y(t)-cparm(1)-arparm(1)*y(t-arorder(1))-... -maparm*res1(t-maorder(1))-... -muparm(1)*dsqrt(res2(t-muorder(1)))-... -xparm(1)*x1(t-xorder(1))-... If a transfer function is not selected: xparm(i)=0 ******************************************************** If :nosqrt is set the first moment equation when a GARCH-M model is estimated is: res1(t)=y(t)-cparm(1)-arparm(1)*y(t-arorder(1))-... -maparm*res1(t-maorder(1))-... -muparm(1)*(res2(t-muorder(1))-... -xparm(1)*x1(t-xorder(1))-... ******************************************************** ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Default Second Moment equation: ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ res2(t)=cparm(2)+gmaparm(1)*(res1(t-gmaorder(1))**2) + garparm(1)* res2(t-garorder(1) + ... ********************************************************** If :igarch is set, the second moment equation is: res2(t)=cparm(2)+(1.0-garparm(1))*(res1(t-gmaorder(1))**2) + garparm(1)* res2(t-garaorder(1) ********************************************************** If :egarch is set, the second moment equation is: g(t) =dabs(res1(t)/dsqrt(res2(t))) - dqsrt(2.0d+00/pi()) - vd*res1(t)/dsqrt(res2(t)) res2(t)= dexp(cparm(2) +gar(1)*dlog(res2(t-garorder(1)) +gma(1))*g(t-1)) This is the Nelson (1991) form of EGARCH. The exact form is that used in Rats (2000) User's guide page 352 ********************************************************** If :egarch2 is set, the second moment equation is: g(t) = dabs( res1(t)/dsqrt(res2(t))) -vd*(res1(t)/dsqrt(res2(t)) res2(t) = dexp(cparm(2)+ +gar(1)*dlog(res2(t-garorder(1)) +gma(1))*g(t-1)) This is the S Plus form of EGARCH except for - in front of vd so leverage is seen as + ********************************************************** If :gjr is set, the second moment equation is: res2(t)=cparm(2)+gma(1)*(res1(t-gmaorder(1))**2) +gar(1)* res2(t-garorder(1) + ... if res1(t-gmaorder(1)) > 0.0 and res2(t)=cparm(2)+gma(1) *(res1(t-gmaorder(1))**2) +gma2(1)*(res1(t-gmaorder(1))**2) +gar(1) * res2(t-garorder(1) + ... otherwise. The GJR model is also called the tarch model by Enders (2004) page 141, Note: The GJR (tarch) model adjusts the gma side of the second moment equation. The :tgarch command adjusts the gar part of second moment equation. ********************************************************* If :tgarch is set, the second moment equation becomes Define (mask-) =1 if res1(t-1) < 0 res2(t)=cparm(2)- gar(1)*res2(t-1) +(mask-)*gar2(1)*res2(t-1) + gma(1)*res1(t-1)*res1(t-1) LF =-.5(ln(res2(t))+res1(t)**2/res2(t) Note: Tsay (2002) uses a (mask+). This implementation is to be consistent with :gjr. Note: The GJR (tarch) model adjusts the gma side of the second moment equation. The :tgarch command adjusts the gar part of second moment equation. The :tgarch is may be very hard to estimate due to the cinstraints imposed. Both :tgarch and :gjr are special cases of :etgarch. ************************************************** If :tgarch2 is set then res1(t)=y(t)-..cparm(1)-ar(i)*y(t-i)..-ma(i)*res1(t-i) res2(t)=cparm(2)+ gar(1)*res2(t-1) + gma(1)*res1(t-1) if res1(t-1) > 0 + gma2(1)*res1(t-1) if res1(t-1) < 0 LF =-.5*(ln(res2(t)**2)+(res1(t)**2/res2(2)**2)) =- ln(res2(t) -.5*(res1(t)**2/res2(2)**2) This is the Zakoian (1994) form of asymetric tar model. Note res2(t)= the square root of the second moment variance. ************************************************** If tgarch3 is set then res1(t)=y(t)-..cparm(1)-ar(i)*y(t-i)..-ma(i)*res1(t-i) res2(t)=cparm(2) + gar(1)*res2(t-1) + gma(1)*abs(res1(t-1))) LF =-.5*(ln(res2(t)**2)+(res1(t)**2/res2(2)**2)) =- ln(res2(t) -.5*(res1(t)**2/res2(2)**2) This is the Zakoian (1994) form of symetric tar model. Note res2(t)= the square root of the second moment variance. ********************************************************* If :etgarch is set then res1(t)=y(t)-..cparm(1)-ar(i)*y(t-i)..-ma(i)*res1(t-i) res2(t)=cparm(2)+gar(1)* res2(t-1) +gma(1)*(res1(t-1)**2)) Where res1(t-1) < 0.0 add terms +cparm(3) +gar2(1)*(res2(t-1)) +gma2(1)*(res1(t-1)**2) LF =-.5(ln(res2(t))+res1(t)**2/res2(t) Notes: This model is discussed in Tsay (2002) page 168. Tsay at that location uses a (mask+). This implementation is to be consistent with :gjr. Higher order terms have been removed to simplify notation. If overflows occur the parameters of the model may have to be restricted in such a way that they do not get near 0.0 during the solution iterations. Sample Jobs Job 1. GARCHEST Joint GARCH(0,1) Estimation b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; call loaddata; call garchest(res,arch,gasout,func,2,n :nar 2 :ngma 1 :print ); b34srun; -------------------------------------------------- Job 2. GARCHEST_2 Joint GARCH(1,1) Estimation b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; call loaddata; call garchest(res,arch,gasout,func,2,n :nar 2 :ngma 1 :ngar 1 :print); b34srun; ---------------------------------------------------- Job 3. More complex model requiring starting values. b34sexec options ginclude('b34sdata.mac') member(wpi); b34srun; b34sexec matrix ; call loaddata; * See Enders page 155 ; j=norows(pi); call olsq(pi pi{1} :print); arch = array(j:); res = array(j:); call garchest(res,arch, pi,func,1,n :maorder idint(array(:1,4)) :nar 1 :arparms array(:%coef(1)) :ngar 1 :ngma 1 :maxfun 2000 :maxg 2000 :maxit 10000 :gradtol .1e-4 :cparms array(:%coef(2),%resvar) :print); call print(sumsq(res)); call tabulate(res,arch); b34srun; --------------------------------------------------- Job 4. Fattails GARCH(1,1) see job GARCHEST_6 Simplex used for starting values b34sexec options ginclude('b34sdata.mac') member(garchdat); b34srun; b34sexec matrix ; call loaddata; * GARCH setup where use fattail; j=1; i=integers(j,norows(sp500)); y=sp500(i); vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); /$ un comment to see effect /$ arch=arch*0.0; call garchest(res,arch,y,func,1,n :lower array(5:.1d-6,.1d-6, 0.,0.0, 0.0) :cparms array(2:mean(y),vstart) :ngar 1 :ngma 1 :maxfun 200000 :maxit 200000 :maxg 200000 :fattail 50. :simplex :print2 :print ); call graph(goodrow(res)); call graph(goodrow(arch)); call tabulate(%resobs,res,arch); b34srun; --------------------------------------------------- Job 5. IGARCH(1,1) Done two ways. See job GARCHEST_7 b34sexec options ginclude('b34sdata.mac') member(garchdat); b34srun; /$ Job illustrates problems in GARCH estimation. /$ IGARCH(1,1) done two ways b34sexec matrix ; call loaddata; * GARCH setup where use fattail; j=1; i=integers(j,norows(sp500)); y=sp500(i); vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); /$ un comment to see effect /$ arch=arch*0.0; call garchest(res,arch,y,func,1,n :lower array(3:0.0, 0.0 , 0.0 ) :upper array(3:1. , 1.d+6 , 1.d+6) :cparms array(2:mean(y),vstart) :ngar 1 :maxfun 200000 :maxit 200000 :maxg 200000 :igarch :simplex :print2 :print ); call graph(goodrow(res)); call graph(goodrow(arch)); call tabulate(%resobs,res,arch); b34srun; /$ IGARCH(1,1) using NLPMIN1 - shows general case. /$ More than IGARCH(1,1) can be done!!! /$ /$ Note that SE are not available b34sexec matrix ; call loaddata; y=sp500; vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); call print('mean y ',mean(y):); call print('vstart ',vstart :); program test; call garch(res,arch,y,func,1,nbad :gar array(:gar) idint(array(:1)) :gma array(:gma) idint(array(:1)) :constant array(:a0 b0) ); if(%active(1)) g(1)=gar+gma-1.; func=(-1.)*func; return; end; call print(test); call echooff; call NLPMIN1(func g :name test :parms gar gma a0 b0 :ivalue array(:.5,.5,mean(y),vstart) :nconst 1 0 :lower array(: 1.d-6, 1.d-6, 1.d-6, 1.d-6) :upper array(: 1.d+2, 1.d+2, 1.d+2, 1.d+2) :print :maxit 100 :iprint final); b34srun; --------------------------------------------------------------- Job 6. EGARCH(1,1). See GARCHEST_8 job b34sexec options ginclude('b34sdata.mac') member(garchdat); b34srun; /$ Job illustrates problems in EGARCH estimation. b34sexec matrix ; call loaddata; * GARCH setup where use egarch ; * Note that for starting values for second moment we use dlog(vstart) ; * For difficult problems the upper limit array caps values such that exp( ) does not blow up; j=1; i=integers(j,norows(sp500)); y=sp500(i); vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); /$ un comment to see effect /$ arch=arch*0.0; call garchest(res,arch,y,func,1,n :lower array(5:.1d-6,.1d-6,.1d-6,0.0,0.0) :upper array(5:10., 10., 20.,1.d+10, 1.d+10) :cparms array(2:mean(y),dlog(vstart)) :ngar 1 :ngma 1 :maxfun 200000 :maxit 200000 :maxg 200000 :egarch .1 :simplex :print2 :maxit2 600 :print ); call graph(goodrow(res)); call graph(goodrow(arch)); call tabulate(%resobs,res,arch); b34srun; --------------------------------------------------- Job 7. GJR GARCH(1,1)/ See job GARCHEST_9 b34sexec options ginclude('b34sdata.mac') member(garchdat); b34srun; /$ Job illustrates problems in GARCH estimation. b34sexec matrix ; call loaddata; * GARCH setup where use GJR Model ; j=1; i=integers(j,norows(sp500)); y=sp500(i); vstart=variance(y-mean(y)); arch=array(norows(y):)+ vstart; res= y-mean(y); /$ un comment to see effect /$ arch=arch*0.0; call garchest(res,arch,y,func,1,n :lower array(5:.1d-6,.1d-6,.1d-6,0.0, 0.0) :cparms array(2:mean(y),vstart) :ngar 1 :garparms array(:.1) :ngma 1 :gmaparms array(:.22) :maxfun 200000 :maxit 200000 :maxg 200000 :gjr array(:.9) :simplex :print2 :maxit2 700 :print ); call graph(goodrow(res)); call graph(goodrow(arch)); call tabulate(%resobs,res,arch); b34srun; --------------------------------------------------- Job 8. ETGARCH Model. See GARCHEST15 job b34sexec options ginclude('b34sdata.mac') member(lee4); b34srun; b34sexec matrix ; call loaddata; * The data generated by GAUSS $ * a1 = GMA = 0.09 $ * b1_n = GAR = 0.5 ( When Negative) $ * b1 = GAR = 0.01 $ call echooff ; maxlag=2 ; y=doo1 ; * y=y-mean(y) ; v=variance(y) ; arch=array(norows(y):) + v; * GARCH on a TGARCH Model ; call garchest(res,arch,y,func,maxlag,n :ngar 1 :garparms array(:.1) :ngma 1 :gmaparms array(:.0001) :maxit 2000 :maxfun 2000 :maxg 2000 /$ :steptol .1d-14 :cparms array(2:.0001,.0001) :print ); * ETGARCH on a TGARCH Model ; * arch=array(norows(y):) + dsqrt(v); call garchest(res,arch,y,func,maxlag,n :ngar 1 :garparms array(:.6) :ngma 1 :gmaparms array(:.1) :etgarch array(:.4,.1, .0001) :simplex :print2 :lower array(:.1e-7,.1e-8,.1e-8,-.01, .001, .001, -.1) :maxit 2000 :maxfun 2000 :maxg 2000 /$ :steptol .1d-4 :cparms array(2: .01,.01) :print ); b34srun ; --------------------------------------------------- Job 9. GARCHEST_A Transfer Function Model b34sexec options ginclude('gas.b34'); b34srun; /$ /$ Estimate a GARCH transfer function. /$ For a direct example using GARCH see GARCH_4 example /$ results tested with RATS in GARCH_4 /$ b34sexec matrix ; call loaddata; call olsq(gasout gasout{1 to 2} gasin{1} gasin{3} :print); /$ res =array(norows(gasout):); /$ res=gasout-mean(gasout); /$ arch=array(norows(gasout):) +%resvar ; call garchest(res,arch,gasout,func,3,n :nar 2 :arparms array(:%coef(1) %coef(2)) :ngma 1 :ngar 1 :xvar gasin array(:%coef(3) %coef(4)) idint(array(:1 3)) idint(array(:2)) :cparms array(:%coef(5), %resvar) :lower array(:-.1d+30,-.1d+30,-.1d+30, -.1d+30, .1d-16, .1d-16, -.1d+30, .1d-16) :maxsteps 4. :gradtol .1e-4 /$ :simplex :print2 :maxit2 2000 :maxit 2000 :maxfun 2000 :maxg 2000 :print); /$ call print(sumsq(goodrow(res)):); call tabulate(res,arch); call graph(goodrow(res)); call graph(goodrow(arch)); b34srun; Notes: :tgarch is very hard to estimate. :tgaropt may need to be used to get just the right form or :limits may have to be used to "turn off" specific constraints. Job GARCHEST_B shows a GARCH transfer function where there are no second order terms. This is tested against the OLSQ command. Job GARCHEST_C shows a GARCH-M transfer function model. To fully exploit the capability of the GARCHEST command it is suggested that the test jobs in matrix.mac be studied and run. GET Gets one or more variables from b34s. call get(name); Obtains the variable name from the current B34S dataset. The alternative call loaddata; gets all variables. Options :dropmiss - allows series to be subset call get(x1, x2 :dropmiss); Warning: If there are multiple calls to get in the same matrix command and there are missing data values in the B34S dataset that has been loaded, then there is the possibility that series will not be aligned. To avoid this problem use only one call get( ) statement per matrix command. Example: b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call get(gasout,gasin); call names; call graph(gasout); b34srun; GETDMF Gets a data from a b34s DMF file. call getdmf(:file 'file name' options); Loads series from a b34s dmf file. Options: :file kk => default 'myruns.dmf' :browse => List what is in file :browsename => Lists names as well as members. :member kk => Sets member to read. Default => read first member; :series nn => Lists series to read. If series is not present, all series are loaded. nn is usually set as a namelist i. e. :series namelist(income, wages) :print => Print what is loaded. Note: The namelist %series contains the series loaded. Examples: call getdmf; call getdmf(:file 'myrun1.dmf'); call getdmf(:file 'myrun1.dmf' :series namelist(x,y,z)); call getdmf(:browse :print); Extended Examples: Loads series in DMF file b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2); call getdmf(browse); call cleardat; call getdmf; call names(all); b34srun; Loads series in a DMS File - Bigger esample b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call echooff; call loaddata; call names(all); call olsq(lnq lnk lnl lnrm1 :print); people=namelist(houston,diana,will,melissa,bobby); call makedmf(lnq,lnl,lnk,lnrm1,lnrm2 %res %y %yhat people :file 'testdmf.dmf' :header 'RES (1972) data' :member test_dat :print); nrow=10; ncol=8; x=rn(matrix(nrow,ncol:)); call print(x); means=array(ncol:); do i=1,ncol; means(i)=mean(x(,i)); enddo; call print(means); call makedmf(x :print :file 'testdmf.dmf' :add :header 'Random Matrix Data'); call getdmf(:browse :browsename :file 'testdmf.dmf'); call cleardat; call getdmf(:file 'testdmf.dmf'); call names(all); call getdmf(:file 'testdmf.dmf' :member data1 :print); call names(all); test=catcol(m1col__1,m1col__3); call print(' ':); call print('mean(test(,1)),mean(test(,2))':); call print( mean(test(,1)),mean(test(,2))); test=catcol(lnq,lnl,lnk); call print(' ':); call print('mean(lnq),mean(lnl),mean(lnk)':); call print( mean(lnq),mean(lnl),mean(lnk)); b34srun; b34sexec options open('testdmf.dmf') unit=62; b34srun; b34sexec dmf infmt=formatted inunit=62$ browse listnames$ b34srun$ b34sexec data file('testdmf.dmf') dmfmember(test_dat) filef= dmf; b34srun; b34sexec list iend=10; b34srun; b34sexec data file('testdmf.dmf') dmfmember(data1 ) filef= dmf; b34srun; b34sexec list; b34srun; GETKEY Gets a key Gets a key code call getkey(i); sets i if a key has been hit call getkey(i:); waits for a key to be hit Codes returned Key/Event Code --------- ----- Backspace ( 8) KeyBackSpace Tab ( 9) KeyTab Return ( 13) KeyReturn Escape ( 27) KeyEscape Printable ASCII chars 32 -> 126 Delete (127) KeyDelete cursor up/down/right/left (128) KeyCursorUp -> (131) KeyCursorLeft shift/cursor u/d/r/l (132) KeyPageUp -> (135) KeyPageLeft ctrl /cursor u/d/r/l (136) KeyUpExtreme -> (139) KeyLeftExtreme Home / Find (140) KeyHome End/Copy / Select (141) KeyEnd Insert / Insert Here (142) KeyInsert Delete-under-cursor (e.g. Remove) (143) KeyDeleteUnder Backtab (e.g. shift/Tab) (144) KeyShiftTab keypad keys 0 to 9 (145) Keypad0 -> (159) Keypad9 keypad - (minus) (160) KeypadMinus keypad . (period) (161) KeypadPoint keypad + or , (plus/comma) (162) KeypadPlus keypad / (slash) (163) KeypadDivide keypad * (asterisk) (164) KeypadMultiply keypad # (hash) (165) KeypadHash keypad Enter (166) KeypadEnter Print (170) KeyPrint function keys 1 to 20 (171) KeyF1 -> (190) KeyF20 SHIFT/fn keys 1 to 20 (191) KeyShiftF1 -> (210) KeyShiftF20 CTRL /fn keys 1 to 20 (211) KeyCtrlF1 -> (230) KeyCtrlF20 ALT /fn keys 1 to 20 (231) KeyAltF1 -> (250) KeyAltF20 Left mouse button down (251) LeftButtonDown Middle mouse button down (252) MiddleButtonDown Right mouse button down (253) RightButtonDown Left mouse button up (254) LeftButtonUp Middle mouse button up (255) MiddleButtonUp Right mouse button up (256) RightButtonUp Mouse movement event (257) MouseMove reserved (258) Graphics window expose/resize event (259) ResizeEvent Close-window request (260) CloseRequest 8-bit ASCII chars 128-255 384 -> 511 (i.e.256+8-bit code) Alt/backspace (520) KeyAltBackspace Alt/tab (521) KeyAltTab Alt/Return (525) KeyAltReturn Alt/Escape (539) KeyAltEscape Alt/0 - Alt/9 560 -> 569 (i.e. 512+ASCII code) Alt/A - Alt/Z 577 -> 602 Example: b34sexec matrix; call echooff; i=0; start continue; call getkey(i); if(i.ne.0)then; call outstring(1,3,'Hit escape to terminate'); call outstring(1,4,'key'); call outinteger(22,4,i); if(i.eq.27)go to stop; endif; go to start; stop continue; b34srun; GETMATLAB Gets data from matlab. call getmatlab(x :file 'junk'); Reads a special file that the b34s supplied matlab m file makeb34s makes. Files created with makeb34s can be read back into MATLAB with the matlab command getb34s. If :file is not present, the default name is _b34smat.dat The commands getmatlab & makematlab pass series as a matrix. If more accuracy is desired the matrix language commands gmatlab and mmatlab (which are shown in the WRITE2 and READ2 examples) can be modified. If accuracy is increased, the matlab m files getb34s.m and makeb34s.m will have to be changed. The MATLAB sentence under the PGMCALL command allows passing of b34s data to MATLAB via vectors. In addition MATLAB commands can be appended. To get data from MATLAB to b34s use the GETMATLAB command under the MATRIX command and use the MATRIX command MAKEDATA to make the appropriate b34s data loading step so that procedures can be run.. Options: :file ' ' - Supply file name. For a related command see makematlab. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; /$ When using the MATLAB GETB34S file use full path /$ xx=getb34s('c:\junk\junk.ttt'); call loaddata; call names; xx=rn(matrix(5,5:)); call makematlab(gasout,gasin:file 'junk.ttt'); call makematlab(xx :file 'junk2.ttt'); call getmatlab(x, :file 'junk.ttt'); call getmatlab(xx2 :file 'junk2.ttt'); call print(x,xx,xx2); call names; cx=complex(xx,xx*2.); call makematlab(cx :file 'junk3.ttt'); call getmatlab(cx2, :file 'junk3.ttt'); call print(cx,cx2); b34srun; GET_FILE Gets a File Name Get a file name using a menu. call get_file(cc); Gets a File name in CC. Example: b34sexec matrix; call get_file(cc); call print('File found was ',cc); call erase(cc); b34srun; GET_NAME Gets a Variable name subroutine get_name(nn,ii); /; /; nn = name /; ii = 0 is a problem /; = 1 otherwise /; /; ***************************************** /; Note: This command has to be loaded with the command call load(get_name); Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; back continue; call loaddata; call load(get_name); call load(dataview); call load(data_acf); call get_name(cc,ii); if(ii.eq.0)go to done; call character(nn,cc); call dataview(eval(cc),nn); go to back; endif; done continue; b34srun; GETRATS Reads RATS Portable file. call getrats(' '); Loads data from rats portable file. If no arguments are passed, the default name of myrun.por is used. Unless keepmiss is in effect, missing data will be removed as the default. Options: :keepmiss - Optionally keep missing data. Variables created: %names => list of series loaded For a related command see makerats. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; newgasi=gasin; newgaso=gasout; call makerats(gasin,newgasi,gasout,newgaso :file 'full.por'); call print(mean(gasin) mean(newgasi) mean(newgaso) mean(gasout) ); call cleardat; call getrats('full.por'); call print(mean(gasin) mean(newgasi) mean(newgaso) mean(gasout) ); call names; call tabulate(obsnum,gasin,newgasi,gasout,newgaso); b34srun; GETSCA Reads SCA FSAVE and MAD portable files. call getsca('fname'); Loads SCA fsave 'fname' into the MATRIX command. This assumes that the first member is loaded. The variant call getsca('fname' :member jones); will load member jones. If the series are of different lengths, then series of different lengths are created. Real*8 and character*8 data are supported. Optional commands :mad allows reading of a SCA MAD file with possibly unequal data lengths. The following options are only needed for last datasets read with more than one series on the row. :maxseries n1 Set to read more than 300 series. The default # of series read in a mad file with more than one series on a row is 300. :maxcwork n2 Set greater than the default of 4000 if parse space is needed. :maxptokens n3 Set to the max parse tokens in file. Can be set above default of 3000. :info Displays internal settings. :testreturn Sets %getsca = 1 if problems loading a member. This switch should not be used unless its value is tested in the next step to stop execution if an error was found. For related commands see makesca and makemad. Variables created: %names => list of names Example: b34sexec options ginclude('b34sdata.mac') member(gas); b34srun; b34sexec matrix; call loaddata; call makemad(gasin,gasout :file 'full.mad' :member test); b34srun; b34sexec matrix; call getsca('full.mad' :mad); call names; call print('mean(gasin)' , mean(gasin) :line); call print('mean(gasout)', mean(gasout) :line); call tabulate(gasout,gasin); b34srun; Character*8 support b34sexec matrix; g(1)='bill'; g(2)='liu'; call makemad(g :file 'full.mad' :member test); b34srun; b34sexec matrix; call getsca('full.mad' :mad); call names; call print(g); b34srun; b34sexec scaio readsca file('full.mad') dataset(test); b34srun; b34sexec list; b34srun; GMFAC - LU factorization of n by n matrix call gmfac(x,l,u); call gmfac(x,l,u,info); Factors n by n matrix x such that x = L*U U = upper triangular matrix L is a "psychologically lower triangular matrix" (i.e. a product of lower triangular and permutation matrices) in L. This command works the same way as the MATLAB lu(x) command. In contrast to x=inv(xx); which uses LINPAC DGECO/DGEFA/DGEDI ZGECO/ZGEFA/ZGEDI unless options are set, GMFAC uses the LAPACK routines DGETRF and ZGETRF. GMFAC optionally will return info > 0 if U(i,i)=0. x - n by n matrix l - Lower triangular matrix u - Upper triangular matrix info - 0 all ok ne 0 x not full rank Example: b34sexec matrix; * Problem from MATLAB; x=matrix(3,3:8. 1. 6. 3. 5. 7. 4. 9. 2.); call gmfac(x,l,u,info); call print(x,l,u,info,l*u); cx=complex(x,dsqrt(dabs(x))); call gmfac(cx,cl,cu,info); call print(cx,cl,cu,info,cl*cu); b34srun; GLM - Generalized Linear Models using Coordinate Descent call glm(y x1 x2 :reg); call glm(y x1 x2 :class m); Implements regularized Path for Generalized Linear Models via coordinate desent. Required: y => left hand variable x1 => right hand variable. Can be a matrix. The format x1{m1 to m2} is supported where m1 can be < 0 for leads. Options: :print :cua => sets covariance updating algorithm. This is the default. :na => sets naive algorithm. :parm r1 => Must be in range 0.0 LE r1 LE 1.0 r1 = 0.0 => ridge model r1 = 1.0 => lasso default = 1.0 :nstdvar => Analysis is not performed on standardized predictor variables. This option is not recommended. :rel_penalty array( ) => Optionally sets relative penalty of each predictor variable. Default => no relative penalty :ne i4 => Sets maximum number of variables allowed to enter the largest model. Default = k. :nlam i4 => Sets number of lamda values. Default = 10 :lamdamin r8 => Sets minimum lamda. Default = .1. :thr r8 => Convergence threshold for each lamda solution. Iterations stop when the maximum standardized coefficent change from the previous iteration is less than th. Default=1.d-4. :reg => Estimate regression model. This is the default. :savex => Saves the X matrix in %X. :sample mask - Specifies a mask real*8 variable that if = 0.0 drops that observation. The :sample option cannot be used if the :outlier or :rr options are used. Unless the mask is the number of obs after any lags, an error message will be generated. The sample variable must be used with great caution when there are lags. A much better option is :holdout. :holdout n - Sets number of observations to hold out. This is useful for model validation purposes. :savemodel - Saves estimated model in object %smod unless reset with :modname. Options for :reg :reg => Estimate regression model. As lamdamin decreases sumsq(%res) converges to OLS sumsq( %res) since there is no shrinkard penalty. Options for :class m models :class m => Perform classification estimation where y has m classes.. :nr => Newton-Raphson :m_nr => Modified Newton-Raphson. This is the default. :maxit i => Number of iterations. Default = 200. Automatic variables: %YVAR - Name of left hand variable. %NAMES - Names of exogenous variables. %LAG - Lag of exogenous variable. %NLP - Total passes over the data summed over all lamda values. %LMU - Actual number of lamda values. %K - # of right hand side variables. %NOB - # of observations. %AO - Constants for all %LMU models. %COEF - Coefficients. For :reg %COEF is (%K,%LMU) For :class %coef is (%K,%K,%LMU) %ALM - Lamda values considered. %Y - Y data. %X - nob by k matrix of data without the constant if :savex supplied. For :reg option: %YHAT - Predicted Y. %YHAT is (%NOB,%LMU) %RSS - Residual sum of sq. %RSS is (%LMU) %RSQ - R**2. %RSQ = (%LMU) %TSS - Total sum of squares. %SUMRE - Sum absolute residuals. %SUMRE is (%LMU) %REMAX - Maximum absolute residual. %REMAX is (%LMU) %RESVAR - Residual Var. %RESVAR is (%LMU) For :class option: Overview: For :reg option for a range of lamda values the lasso model min(beta) sum(i=1,N) (y(i)-beta(0)-sum(j=1,p)x(i,j)*beta(j))**2 + lamda*sum(j=1,p)abs(beta(j) is solved. The objectyive is to shrink the number of regressors. As lamda approaches 0.0, the model approaches OLS. For :class k option L1 penalized logictic and multinomial regresison mnodels are fit. For these more complex models see the below lsited references. Reference: Hastie-Tibshirani-Friedman. "The Elements of Statistical Learning." (2009) pages 126, 661-668. Friedman, Jerome. Trevor Hastie and Rob Tibshirani. "Regularized Paths for Generalived Linear Models via Coordinate Descent" technical report 29 April 2009 Stanford University. Code for this command was originally developed by Friedman- Hastie-Tibshirani under the GPL-2 license for R. Changes and improvements have been made to implement into B34S. Note: This is an experimental command. The :reg option has been tested against the Matlab implementation. The :class option has not yet been validated. Examples: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; m=5 ; nlam=20; call olsq(gasout gasout{1 to m} gasin{1 to m} :print); olsres =%res; olsyhat=%yhat; call glm(gasout gasout{1 to m} gasin{1 to m} :print :nlam nlam :thr 1.d-8 :na :lamdamin .3d-5 ); call print(%coef); call print(%nin,%ia); call tabulate(%a0 %rsq %rss %alm); glmyhat=%yhat(,%lmu); glmres = %res(,%lmu); call tabulate(%y,olsyhat,olsres,glmyhat,glmres); call print('rss = ',sumsq(glmres)); call graph(olsres glmres :nolabel); call graph(%y olsyhat glmyhat :nolabel); call olsq(gasout gasout{1} gasin{3 to 4} :print); b34srun; b34sexec matrix; call echooff; idotest2=1; idotest3=1; idotest4=1; ido_olsq=1; ido_reg =0; ido_rf =1; call restore(:file 'c:\b34slm\lib\glmdata.psv' :list); call restore(:file 'c:\b34slm\lib\glmdata.psv'); call printall; if(idotest2.ne.0)then; if(ido_olsq.ne.0)call olsq(y2 x2 :print); call glm(y2 x2 :lamdamin .0005 :print); endif; /; +++++++++++++++++++++++++++++++++++++++++++++++++++++++ if(idotest3.ne.0)then; if(ido_olsq.ne.0)call olsq(y3 x3 :print); if(ido_reg.ne.0)then; call glm(y3 x3 :print :lamdamin .03 :savex); %alm=dropfirst(%alm,1); %rss=dropfirst(%rss,1); %rsq=dropfirst(%rsq,1); call tabulate(%rsq,%rss,%alm); call graph(%alm %rss :plottype xyplot :heading 'sumsq(%res) vs lamda'); call graph(%alm %rsq :plottype xyplot :heading '%rsq vs lamda'); endif; if(ido_rf.ne.0) call ranforest(y3 x3:class 42 :print :vote_yhat); call glm(y3 x3 :class 4 :print :nlam 50); call print(%coef,%alm %y %yhat); call names(all); endif; /; ++++++++++++++++++++++++++++++++++++++++++++++++++++++ if(idotest4.ne.0)then; if(ido_olsq.ne.0)call olsq(y4 x4 :print); if(ido_rf.ne.0) call ranforest(y4 x4 :class 2 :print :vote_yhat); if(ido_reg.ne.0)call glm(y4 x4 :print :lamdamin .0003); call glm(y4 x4 :class 1 :print :lamdamin .0003); do i=1,norows(%yhat); do j=1,nocols(%yhat); if(%yhat(i,j) .ge. .5)%yhat(i,j)=2.; if(%yhat(i,j) .lt. .5)%yhat(i,j)=1.; enddo; enddo; call tabulate(%y %yhat(,1),%yhat(,2), %yhat(,3),%yhat(,4),%yhat(,5), %yhat(,6),%yhat(,7)); test=catcol(%y %yhat); call print(ccf(test),%a0,%alm); endif; b34srun; Testing a GLM Model /; /; Ridge regression also shown /; /; For further detail see Stokes (200x, Chapter 10) /; /; %b34slet dorats =0; %b34slet doridge=0; %b34slet dolasso=0; b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(ridge :staging); call load(lasso :staging); call load(glm_info :staging); call echooff; /; logic works for holdout > 0 k=10; holdout=100; maxlag=k; /; at issue is how are yhat values calculated within the /; sample treated? /; Fore purposse of this analysis they are removed call olsq(gasout gasout{1 to k} gasin{1 to k} :print :savex :holdout holdout); %yfuture= gasout(integers(norows(%x)+maxlag,norows(gasout))); olsf=mfam(%xfuture)*vfam(%coef); olsfss=sumsq(afam(%yfuture)-afam(olsf)); call print('ols forecast error sum sq ',olsfss:); nlam=100; lam_min=.000003; thr =.1e-6 ; parm = .5; ne = 20; call glm(gasout gasout{1 to k} gasin{1 to k} :print :savex :lamdamin lam_min :nlam nlam :holdout holdout :thr thr :parm parm :ne ne); call print(' ':); %yfuture= gasout(integers(norows(%x)+maxlag,norows(gasout))); call glm_info(%yfuture,%xfuture,%coef,%a0,%alm,glmf, fss,mod,1); call print('glm forecast sum of squares ',fss:); res_ols=vfam(%yfuture)-olsf; res_glm=vfam(%yfuture)-glmf; call graph(%yfuture,olsf,glmf :nolabel :nocontact :pgborder :heading 'OLS vs GLM out of sample'); call graph( res_ols res_glm :nolabel :nocontact :pgborder :heading 'OLS vs GLM residual out of sample'); call tabulate(%yfuture,olsf,glmf res_ols res_glm); /; tests on reduction loss rss_glm=array(2*k:); ne_used=array(2*k:); icount=1; do ne=1,2*k; ii=ne; call glm(gasout gasout{1 to k} gasin{1 to k} :savex :lamdamin lam_min :nlam nlam :holdout holdout :thr thr :parm parm :ne ii :print); call print(' ':); %yfuture= gasout(integers(norows(%x)+maxlag,norows(gasout))); call glm_info(%yfuture,%xfuture,%coef,%a0,%alm,glmf,fss, mod,0); rss_glm(icount)=sfam(fss); ne_used(icount)=sfam(dfloat(ii)); icount=icount+1; enddo; rss_glm =dropfirst(rss_glm,1); ne_used =dropfirst(ne_used,1); call tabulate(rss_glm,ne_used); call graph(ne_used rss_glm :grid :heading 'Loss in RSS due to model reduction' :plottype xyplot :nocontact :nolabel :pgborder); %b34sif(&doridge.ne.0)%then; lamda=2.; lamda=%alm(%lmu); call print('Lamda assumed to be ',lamda); call ridge(%y,%x,lamda,%coef,%names,%lag,ridge_c,0); call ridge(%y,%x,lamda,%coef,%names,%lag,ridge_c,1); %b34sendif; %b34sif(&dolasso.ne.0)%then; /; Redefine %coef call olsq(gasout gasout{1 to k} gasin{1 to k} :savex :holdout holdout); lamda=10.*sum(%coef)/2.0; call echooff; call lasso(%y,%x,%coef,%lcoef1,%l_t1,lamda,lresid1,1); call lasso(%y,%x,%coef,%lcoef2,%l_t2,lamda,lresid2,3); call tabulate(%names,%lag,%coef,%se,%t,%lcoef1, %l_t1,%lcoef2,%l_t2); %b34sendif; b34srun; %b34sif(&dorats.ne.0)then; b34sexec options open('rats.dat') unit(28) disp=unknown$ b34srun$ b34sexec options open('rats.in') unit(29) disp=unknown$ b34srun$ b34sexec options clean(28)$ b34srun$ b34sexec options clean(29)$ b34srun$ /; Uses logic from RATS User's Guide Version 6.1 Page 192 b34sexec pgmcall$ rats passasts pcomments('* ', '* Data passed from B34S system to RATS', '* ', "display @1 %dateandtime() @33 ' Version '%ratsversion()" '* ') $ PGMCARDS$ * * Non centered results * cmoment # constant gasout{1 to 6} gasin{1 to 6} gasout linreg(cmoment) gasout # constant gasout{1 to 6} gasin{1 to 6} do row=1,13 compute %cmom(row,row)=%cmom(row,row)+2 end do row linreg(cmoment) gasout # constant gasout{1 to 6} gasin{1 to 6} b34sreturn$ b34srun $ b34sexec options close(28)$ b34srun$ b34sexec options close(29)$ b34srun$ b34sexec options /$ dodos(' rats386 rats.in rats.out ') dodos('start /w /r rats32s rats.in /run') dounix('rats rats.in rats.out')$ B34SRUN$ b34sexec options npageout WRITEOUT('Output from RATS',' ',' ') COPYFOUT('rats.out') dodos('ERASE rats.in','ERASE rats.out','ERASE rats.dat') dounix('rm rats.in','rm rats.out','rm rats.dat') $ B34SRUN$ %b34sendif; GMINV - Inverse of General Matrix using LAPACK call gminv(x,xinv); call gminv(x,xinv,info); call gminv(x,xinv,info,rcond); Inverts a general matrrix using LAPACK Given x - is n by n matrix xinv - inverse of x Optional Arguments. If these arguments are present program will not stop if there is a problem. This allows "code trapping" of a problem. info - 0 all ok ne 0 x not full rank rcond - if present returns the condition of the matrix if info = 0 GMIN uses LAPACK DGETRF/ZGETRF and DGETRI/ZGETRI DGECON/ZGECON Note that rcond takes time to compute. The test job GMINV_2 suggests that LINPACK runs faster than LAPACK. The advantage of GMINV is that rank problems can be easily captured and code can be written to handle the exception. Rank can also be found using PINV(x,rank) for a real*8 matrix. Example 1: b34sexec matrix; n=5; x=rn(matrix(n,n:)); call gminv(x,xinv1,info); xinv2=inv(x); test1=x*xinv1; test2=x*xinv2; call print(x ,xinv1 ,xinv2 ,test1,test2); cx=complex(x,dsqrt(dabs(x))); call gminv(cx,cxinv1,info); cxinv2=inv(cx); test1=cx*cxinv1; test2=cx*cxinv2; call print(cx,cxinv1,cxinv1,test1,test2); b34srun; Example 2: Speed tests of LAPACK vs LINPACK b34sexec matrix; * Tests speed of Linpack vs LAPACK; call echooff; icount=0; n=0; upper=600; mesh=50; top continue; icount=icount+1; n=n+mesh; if(n .eq. upper)go to done; x=rn(matrix(n,n:)); ii=matrix(n,n:)+1.; call timer(base1); call gminv(x,xinv1,info); call timer(base2); error1(icount)=sum(dabs(ii-(xinv1*x))); call timer(base3); xinv1=inv(x); call timer(base4); error2(icount)=sum(dabs(ii-(xinv1*x))); size(icount) =dfloat(n); lapack(icount) =(base2-base1); linpack(icount)=(base4-base3); call free(x,xinv1,ii); call compress; go to top; done continue; call tabulate(size,lapack,linpack,error1,error2); call graph(size lapack,linpack :plottype xyplot); b34srun; GMSOLV - Solve Linear Equations system using LAPACK call gmsolv(x,b,ans,info); Solves a linear system using LAPACK Given x - n by n matrix b - n by k matrix ans - answer of inv(x)*b Optional Argument: If this argument is present the program will not stop if there is a rank problem. info - 0 all ok ne 0 not full rank :refine If present the LAPACK routines DGESVX and ZGESVX will be used to refine the solution. This will take substantially more time. Automatic variables created include: %rcond = LAPACK estimate of the condition %ferror = forward error %berr = backward error :refinee If present the LAPACK routines DGESVX and ZGESVX will be used to refine the solution. This will take substantially more time. Automatic variables created include: %rcond = LAPACK estimate of the condition %ferror = forward error %berr = backward error With this option matrix equilibrating will be performed. If :refine or :refinee are used, info must be present and tested. Routine will not stop if there are problems. GMSOLVE uses LAPACK DGETRF/ZGETRF and DGETRS/ZGETRS. Since the inverse is NOT formed, and BLAS 3 is used for large systems there is a speed gain over an implementation that explicitly forms the inverse. Since the command solves x * ans = b Assuming x is (n by n) if b is an identity matrix, then here ans will be the inverse. This tricks the program to give the inverse but is wasteful of space. If :refine is used, then the inverse may be accurate, but at higher cost!! For higher accuracy, consider using inv( ) with real*16 data or a VPA implementation. Example: b34sexec matrix; n=5; x=rn(matrix(n,n:)); b=rn(x); call gmsolv(x,b,test1,info); test2=inv(x)*b; call print(x ,b ,test1,test2); cx=complex(x,dsqrt(dabs(x))); cb=complex(b,dsqrt(dabs(b))); call gmsolv(cx,cb,test1,info); test2=inv(cx)*cb; call print(cx,test1,test2); b34srun; GRAPH High Resolution graph. call graph(x); Allows graphing of one to nine series using high resolution plotting. If graphics not available, use plot command. The graph command supports real*8, real*4 and integer*4 1d objects. Advanced Graph features using keywords: :heading 'Heading here' Can set up to 72 columns. Heading set above xlabeltop. :pspaceon can improve looks. :htitle xsize ysize xsize = 1. => 75 characters per line. ysize = 1. => 25 characters per col. :plottype keyword obsplot - Series plotted against observation number. This is the default. xyplot - Plots first series on x axis. xyzplot - Plots three series in 3D Plot. timeplot - Plots a series with time info with time dates on the x axis. If the series is not a time series timeplot works the same as obsplot. obsplotb - Series plotted with bounts. Data entered as series lower upper. Useful if there are not too many observations. timeplotb- Time series plotted with bounds. Data entered as series lower upper. Useful if there are not too many observations. If the series is not a time series timeplotb works the same as obsplotb. hist2d - 2 dimensional histogram. hist2dv - 2 dimensional histogram with labels. hist2dhl - 2 dimensional histogram with high/low labels. series 1 = low value, series 2 = high value) hist3d - 3 dimensional histogram hist2dc - 2 dimensional cumulative histogram hist3dc - 3 dimensional cumulative histogram bar2d - 2 dimensional bar graph bar2dv - 2 dimensional bar graph with labels bar3d - 3 dimensional bar graph bar2dc - 2 dimensional cumulative bar graph bar3dc - 3 dimensional cumulative bar graph pie - Pie chart scatter - Scatter diagram xyscatter- Scatter diagram with first series on x axis Note: scatter and xyscatter require :markpoint options be set. Note: For a time plot use xyplot and pass a year variable in. The command call loaddata; will generate a julian variable bjulian_ that can be used. See the commands getyear( ) and fyear( ) commands. If a variable has time series info, then makejul( ) can be used to get the julian date vector. The next four options require a 2-D matrix be passed mesh - Plots a 2 by 2 matrix meshc - Plots a 2 by 2 matrix using a matrix of colors. meshstep - Plots a 2 by 2 matrix using a step 3-D plot meshstepc - Plots a 2 by 2 matrix using a step 3-D plot with colors Label with :xlabel 'text here' 'C' :ylabelleft 'text here' 'C' :zlabelleft 'text here' 'C' scale refers to the number x and y coordinates of the matrix not the values. The commands :xscale array(: min(latt) mean(latt) max(latt)) :yscale array(:10. 50. 100.) would only label the y rows 10 50 and 100 and the x rows corresponding to the min(latt) mean(latt) and max(latt) To set the ranges of x and y shown on the plot use :pgunits array(:minx miny maxx maxy) :pgunitsz array(:minz maxz) is usually not needed. **************************************************** The next two options require a 3-D matrix and do volume plots. Vold3 uses a range of 9 colors. Vold3c plots individual cells. vol3d - Plots a three dimensional real*8 matrix passed as a 1-D array. If dimensions are not the same, use :dimension index(n1,n2,n3) to set the dimensions of the matrix. Plots using one color vol3dc - Plots a three dimensional real*8 matrix passed as a 1-D array. If dimensions are not the same, use :dimension index(n1,n2,n3) to set the dimensions of the matrix. Elements of matrix in range 0.0 - 256. plot as a color. For elements = missing plot without color. If :scale is supplied data will be scaled to be in range 0.0 - 256. Example of complex graphs: b34sexec options ginclude('b34sdata.mac') member(windvel); b34srun; b34sexec matrix; call loaddata; call graph(vel :Heading 'Data looked at as a 1-D array'); call graph(vel :plottype vol3d :d3axis :d3border :grid :angle 10. :dimension index(35,41,15) :heading 'Vol3d plot of Wind Vel.'); call graph(vel :plottype vol3d :d3axis :d3border :grid :angle 30. :scale :dimension index(35,41,15) :heading 'Vol3d plot of Wind Vel.' ); call graph(vel :plottype vol3dc :d3axis :d3border :grid :scale :dimension index(35,41,15) :angle 10. :heading 'Vol3dc plot of Wind Vel.'); b34srun; **************************************************** The next ten options require 3 series contours - 3D Scatter plot. contoursx- 3D Scatter plot line to X axis. contoursy- 3D Scatter plot line to Y axis. contoursz- 3D Scatter plot line to Z axis. contour3 - Three demensional surface plot. User supplies three vectors. steped3d - Stepped plot version of countour3. contourc - Same as contour3, except that Height vector cheight is generated. steped3dc- Same as contourc except uses step plot contour2 - Two dimensional line based contour plot. contourf - Two dimensional fill based contour plot. obsplotb - Line plot with two error bars. Bars set as 2nd and 3rd series. timeplotb- Time plot with two error bars. Bars set as 2nd and 3rd series. Example: b34sexec matrix; n=100; k=20; x=rn(matrix(n,k:)); call graph(x :plottype mesh :angle 10. :d3axis :d3border :heading 'This is the data'); call graph(x :plottype meshc :heading 'The data as a surface'); x=transpose(x)*x; call graph(x :plottype mesh :heading 'This is what transpose(x)*x is'); call graph(x :plottype meshc :heading 'Transpose(x)*x in color!!'); call graph(x :plottype meshc :grid :heading 'Transpose(x)*x in color with Grid'); call graph(x :plottype mesh :angle 10. :heading 'Transpose(x)*x - angle 10.'); call graph(x :plottype meshc :angle 10. :heading 'Transpose(x)*x - angle 10.'); call graph(x :plottype mesh :rotation 90. :heading 'Transpose(x)*x rotation 90.'); call graph(x :plottype meshc :rotation 90. :grid :d3axis :d3border :heading 'Transpose(x)*x rotation 90.'); call graph(x :plottype meshstep :rotation 70. :angle 10. :grid :heading 'Transpose(x)*x rotation 70. meshstep'); call graph(x :plottype meshstepc :rotation 70. :angle 30. :grid :d3axis :heading 'Trans(x)*x rotation 70. meshstepc'); call graph(x :plottype meshstepc :rotation 70. :angle 0. :grid :heading 'Trans(x)*x rotation 70. meshstepc'); b34srun; ******************************************************* Note: For obsplotb and timeplotb the series are entered as y ylower yupper. The pie graph type requires two series. The second of which must be character for labels. Color keywords black red yellow green cyan blue magenta white gray bred byellow bgreen bcyan bblue bmagenta bwhite ------------------------------------------------------- :overlay keyword keywords recognized are: acfplot - assumes acfplot2d acfplot2d acfplot3d for ACF we use call graph(acfval se :nokey :nolabel :heading 'ACF Plot' :overlay acfplot); Example: b34sexec options ginclude('gas.b34')$ b34srun$ b34sexec matrix; call loaddata; acf2=acf(gasin,24,se2,pacf2); call graph(acf2 se2 :overlay acfplot); b34srun; ---------------------------------------------------------- :file 'file name' Saves plot. :noshow Turns off display of graph if hardcopy to a file is selected :fitspline Fits a spline to line and x-y plots. :linetype key key can be solid (Default) dotted dashed dotdash dotdotdash (device dependent) longshort (device dependent) short (device dependent) :linetype solid dotted makes plot 1 solid and plot 2 dotted :markpoint istart ievery icode1 icode2 istart Sets position to mark point. ievery Sets number between points. icode1 Sets 0 => No not mark the plot (default) 1 => digits 2 => letters 3 => Markers 4 => Symbols icode2 Sets what to plot with For digits set 1-9 For letters 1-26 (A-Z) 27-25 (a-z) For Markers 1-20 For Symbols 33-126 or 161-255 icode2 can be set as an array Examples: 1 1 3 14 => mark with big dot 1 1 4 111 => mark with small dot 1 1 4 116 => mark with dot 1 1 4 120 => mark with a bar 1 1 4 166 => Mark with small dot 1 1 1 index(1,2,3,4,5,6,7,8,9) Recommended settings are 1 1 4 162 1 1 4 165 1 1 4 206 1 1 4 218 1 1 4 219 For a complete visual table of what is available go to "Settings" "Graph Settings" "View Character / Symbol Table" :wait # of centiseconds to wait while a graph displays. :colors keywords for 1-9 :grcharset filename loads a character set file. Once this file is loaded it is the default charset. See call igrcharset('filename') command to set the default character set. File names supported 'H' Hardware font. 'standard.chr' General Purpose character set. 'roman.chr' Times Roman 'romanbld.chr' Roman Bold 'swiss.chr' Swiss / Helvetica style font 'swissbld.chr' Swiss / Helvetica bold font 'simplexrchr' Similar to standard with more detail. 'duplexr.chr' More detailed that simplex. 'triplexr.chr' Heavier variant than duplex. 'complexr.chr' More tapered segments than triplexr. 'complexi.chr' Italic version of Complex Roman. 'triplexi.chr' Italic version of triplex Roman. 'simplexs.chr' Handwritten style. 'complexs.chr' More detailed variant of simplexs. 'simplexg.chr' Greek characters added to simplex. 'complexg.chr' More detailed simplexg. 'gothicen.chr' Very detailed old English style. 'gothicit.chr' Variant of gothicen. 'standden.chr' Danish variant of standard. 'standfra.chr' French variant of standard. 'standger.chr' German variant of standard. 'standita.chr' Italian variant of standard. 'standnor.chr' Norwegian variant of standard. 'standswe.chr' Swedish variant of standard. 'standuk.chr' UK variant of standard. :grcharfont ikey ikey codes Fixed Proportional 1 Helvetica Courier 2 Helvetica ital Courier ital 3 Helvetica bold Courier bold 4 Helvetica bold/Ital Courier bold/ital 5 Times Roman Courier 6 Times Roman ital Courier ital 7 Times Roman bold Courier bold 8 Times Roman bold/ital Courier bold/ital The grcharfont option requires :grcharset 'h' be in effect. Examples of grcharset and grcharfont are in graph7. b34sexec options ginclude('gas.b34')$ b34srun$ b34sexec matrix; call loaddata; call graph(gasout :heading 'This is the current default'); call graph(gasout :heading 'This is a standard.chr' :grcharset 'standard.chr'); call grcharset('H'); call graph(gasout :heading 'This is a test 1' :pspaceon :grcharfont 1 :file 't1.wmf'); call graph(gasout :heading 'This is a test 2' :grcharfont 2 :file 't2.wmf'); call graph(gasout :heading 'This is a test 3' :grcharfont 3 :file 't3.wmf'); call graph(gasout :heading 'This is a test 4' :grcharfont 4 :file 't4.wmf'); call graph(gasout :heading 'This is a test 5' :grcharfont 5 :file 't5.wmf'); call graph(gasout :heading 'This is a test 6' :grcharfont 6 :file 't6.wmf'); call graph(gasout :heading 'This is a test 7' :grcharfont 7 :file 't7.wmf'); call graph(gasout :heading 'This is a test 8' :grcharfont 8 :file 't8.wmf'); call graph(gasout :heading 'This is a test roman.chr' :grcharset 'roman.chr'); call graph(gasout :heading 'This is a test romanbld.chr' :grcharset 'romanbld.chr'); call graph(gasout :heading 'This is a test swiss.chr' :grcharset 'swiss.chr'); call graph(gasout :heading 'This is a test swissbld.chr' :grcharset 'swissbld.chr'); call graph(gasout :heading 'This is a test fixed.chr' :grcharset 'fixed.chr'); call graph(gasout :heading 'This is a test fixedbld.chr' :grcharset 'fixedbld.chr'); call graph(gasout :heading 'This is a test simplexr.chr' :grcharset 'simplexr.chr'); call graph(gasout :heading 'This is a test duplexr.chr' :grcharset 'duplexr.chr'); call graph(gasout :heading 'This is a test triplexr.chr' :grcharset 'triplexr.chr'); call graph(gasout :heading 'This is a test complexr.chr' :grcharset 'complexr.chr'); call graph(gasout :heading 'This is a test H' :grcharset 'H'); call graph(gasout :heading 'This is a test complexi.chr' :grcharset 'complexi.chr'); call graph(gasout :heading 'This is a test triplexi.chr' :grcharset 'triplexi.chr'); call graph(gasout :heading 'This is a test simplexs.chr' :grcharset 'simplexs.chr'); call graph(gasout :heading 'This is a test complexs.chr' :grcharset 'complexs.chr'); call graph(gasout :heading 'This is a test simplexg.chr' :grcharset 'simplexg.chr'); call graph(gasout :heading 'This is a test complexg.chr' :grcharset 'complexg.chr'); call graph(gasout :heading 'This is a test complexc.chr' :grcharset 'complexc.chr'); call graph(gasout :heading 'This is a test gothicen.chr' :grcharset 'gothicen.chr'); call graph(gasout :heading 'This is a test gothicit.chr' :grcharset 'gothicit.chr'); b34srun; :scale Scale all data to have the same mean as the first variable :nolabel Turns off labels :pspaceon Turns on proportional spacing. This setting stays on unless turned off in a later graph call. :pspaceoff Turns off proportional spacing. This setting stays off unless turned on in a later graph call. :rotation real number :nocontact real number Automatically adjusts xaxis and yaxis such that plot does not touch the sides An optional argument percent determines how much adjustment. A manual mode way to get the same result is to do an xyplot and manually supply :setxrange and :setyrange nocontact works with obsplot and xyplot. :angle real number in range 0.0 - 45 :d3axis Sets 3D axis. :d3border Sets 3D border. :dimension Sets dimension of 3 dimensional arrays. form :dimension index(5,6,7) :box ngrid Notes on 3D routines. The commands CONTOUR3, CONTOURC, CONTOUR2 and CONTOURF transform three vectors into a two dimensional matrix of heights with dimensions NGRID by NGRID by the Interacter subroutine iPGXYZToGrid. :grid Turn on grid lines for mesh, meshc, meshstep, meshstepc, vol3d, vol3dc. Turns on Granicules for plots using dotted lines. :nokey Turns off key for plots :noxlabel Turn off X label :noylabel Turn off Y label ************************************************** Advanced Graph Settings to over ride defaults :xdecimal int Sets number of decimal places. < 0 => autoselect. :ydecimal int Sets number of decimal places. < 0 => autoselect. :zdecimal int Sets number of decimal places. < 0 => autoselect. :rxtick real Sets relative tick size. Default = 1.0 :rytick real Sets relative tick size. Default = 1.0 :rztick real Sets relative tick size. Default = 1.0 :xlabeltop ' ' key Sets text up to 90 and key where key must be 'L', 'C' or 'R' for left, centered, or right. :xlabeltop can be used to annotate under heading :heading. :xlabelpos changes position Use :heading to change the size of the title. Use :xlabeltop to write near the top of the graph. :xlabel ' ' key Sets text up to 90 and key where key must be 'L', 'C' or 'R' for left, centered, or right :ylabelleft ' ' key Sets text up to 90 and key where key is a 2 level code set inside ' '. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :ylabelright ' ' key Sets text up to 90 and key where key is a two level code set inside ' '. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :zlabelleft ' ' key Sets text up to 90 and key where key is a 2 level code set inside ' '. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :zlabelright ' ' key Sets text up to 90 and key where key is a two level code set inside ' '. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :xlabelpos r8 Sets relative position of xlabel. r8 must be in range 0 to 1.0 Default = .7. Smaller numbers mean nearer to figure. :ylabelpos r8 Sets relative position of ylabel. r8 must be in range 0 to 1.0 Default = .8. Smaller numbers mean nearer to figure. :zlabelpos r8 Sets relative position of zlabel. r8 must be in range 0 to 1.0 Default = .8. Smaller numbers mean nearer to figure. :linewidth int array of 2 elements Sets line width in pixels for screen and hard copy. Default is :linewidth index(1 1) :xscale real array Sets user x label values. Length of array must be le 100 :xscale array(:4 8 12) :yscale real array Sets user y label values. Length of array must be le 100 :yscale array(:4 8 12) :zscale real array Sets user z label values. Length of array must be le 100 :zscale array(:4 8 12) :histscale int array Sets user histogram label values. Length of array must be le 100 :histscale integers(1,6) :barscale int array Sets user bar label values. Length of array must be le 100 :barscale integers(1,8) :hardcopyfmt key Sets hardcopy output format for this graph only. HP_GL => 1 HP-GL EPS => 2 PostScript RAST => 3 Raster Graphic PCX_BMP => 6 PCX/BMP LOTUS => 7 Lotus PIC DXF => 8 DXF CGM => 9 Computer Graphics Metafile WPM => 10 Windows Print Manager WMF => 11 Windows Meta File HP_GL2 => 12 HP_GL/2 Example: :hardcopyfmt hp_gl :pgaxesxy real array of 2 elements Sets position of axes. Default is 0.0 0.0 :pgunits array(:mmin_1 mmin_2 mmax_1 mmax_2) sets min x min y max x max y for plots. This option is usually needed for mesh, meshc, meshstep and meshstepc plots. :pgunitsz array(:minx maxz) sets min and max of z. This can be used with mesh, meshc, meshstep and meshstepc plots, but is usually not needed. :charsize array(:charwidth charhight) sets character width and height for text output. 1. 1. is the default. These are numbers along graph Alternatives to above commands for expert users. :setxscale real array of 2 elements Sets left hand value and incrument for X scale. If this parameter is not set correctly all or parts of the graph may be off the screen. The value r1 is the lower left X value and r2 is the incrument between tick marks for X scale. :setyscale real array of 2 elements Sets left hand value and incrument for Y scale. If this parameter is not set correctly all or parts of the graph may be off the screen. The value r1 is the lower left Y value and r2 is the incrument between tick marks for Y scale. :nxticks i4 Sets number of user X ticks if :setxscale is in effect. 2 LE i4 LE 100. Default = 5. :nyticks i4 Sets number of user Y ticks if :setyscale is in effect. 2 LE i4 LE 100. Default = 5. :setxrange real array of 2 elements Sets Min (r1) and Max (r2) for xscale. :setyrange real array of 2 elements Sets Min (r1) and Max (r2) for yscale. :grborder Draws a border around graph area. Not used in full screen mode. :pgborder Draws a border around presentation graph area. :pgxscaletop key Places x scale on top. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgxscale key Places x scale on bottom. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgzscale key Places z scale for 3 d graphs. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis Example: The following code places ticks top, middle, bottom and left and right. It is based on fact that :pgxscaletop redefines tick positions. :grunits array(:mmin_1 mmin_2 mmax_1 mmax_2) :pgarea array(:.1 .1 .9 .9) :pgunits array(:mmin_1 mmin_2 mmax_1 mmax_2) :color black :heading title :pgxscale 'NT' :pgaxes :pgxscale 'NT' :pgborder :pgyscaleleft 'NT' :pgyscaleright 'I' :pgxscaletop 'I' :pgxscale 'NT' :pgyscaleleft key Places y scale on left. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgyscaleright key Places y scale on right. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis Examples of GRAPH command for simple plots call graph(x,y,z :plottype hist2d :heading 'Test of Histogram'); call graph(x :heading 'Test of plot' :file 'c:\junk\test.wmf'); call graph(x); Example of xyscatter plot done two ways b34sexec options ginclude('class.mac') member(cars); b34srun; b34sexec hrgraphics plottype=xyscatter imode=3 displayprint gport('c:\junk\myplot.wmf') ; plot=(x,y) nolabel nokey ylabelleft('Value of a Car' 'c9') xlabel('Age of the Car') title('Value vs Age')$ b34srun$ /; Alternative using Matrix Command /; Note :nocontact option. :grid option turned off b34sexec matrix; call loaddata; call graph(x y :plottype xyscatter :ylabelleft 'Value of a Car' 'c9' :xlabel 'Age of the Car' /; :markpoint 1 1 3 33 :markpoint 1 1 3 14 :nocontact :pgborder :heading 'Value vs Age' /; :grid )$ b34srun$ GRAPHP - Multi-Pass Graphics Programing Capability This command is not for the general user. However by its use custom graphic objects can be displayed. call graphp(:start); call graphp(:cont ...); call graphp(:final); The above commands allow users to program complex graphs that are not possible with the "built-in" graphics capability in the GRAPH command. The GRAPHP command is not intended for the general user. A detailed knowledge of Interacter Software is assumed. GRAPHP commands provide access to the Interacter Graphics primative commands so that custom graphics applications can be developed by the B34S programming team. These applications are distributed in the form of B34S Matrix Command subroutines and programs to give the user to the ability to create "custom" graphs without hardwiring the graph types into the B34S execuitable. A general user wishing to make use of this facility for building user custom graphics should license Interacter / Winteracter and use the GRAPHP command to prototype potential graphics applications before they are hard coded in the user's Fortran. Help documentation for the GRAPHP command is terse. Note that while the menu system in Interacter and Winteracter are different, the graphics routines are the same. All GRAPHP command blocks begin with the :start option, contain a number of :cont commands and finish with the :final option. Other matrix commands can be mixed inside the GRAPHP commands as long as GRAPH and GRREPLAY are not called. The reason for this limitation is that such calls would kill the graphics screen. The only exception to this would be to save and restore the screen. This work-around may not function correctly and is not supported. Use of the :toolbox command allows user input into the graph. The Toolbox feature allows the user to interactively build complex graphs that can be saved into *.bmp or *.pcx format which can be imported into Word. On the :cont option, commands are processed in sequence so that for example colors can be changed as we move down a list of options. Colors set by integer value n the range 0-255 rather than names. Base colors can be obtained with the integer function i=icolor(red); and shades can be adjusted by adding or substracting from i. *************************************************************** Missing Data: The MATRIX command dmax and dmin have an optional argument : which supports missing data. The graphp commands :grpoint, :grjoin and :grmarker will ignore missing data. Warning: Many arguments in graphp are not checked due to many possible ways the commands are used. Users have to take care to check the results of their setups. The design goal of GRAPHP is to allow users to develope custom subroutines for types of graphs that are not possible with the GRAPH command. As we get more experience with graphp, the command language for various commands may be changed. The present implementation should be considered to be in "mature" beta form. Bugs may remain. *************************************************************** :start option section :start is the first option in a sequence of GRAPHP commands. The only options allowed on the :start command are :file and :hardcopyfmt :file ' ' Saves the graph in a file. If file is present, the graph will not show on the screen. If a blank string of the form ' ' is passed and B34S is running on windows, the file will be placed on the clip board as long as the file save type is wmf. Due to the fact that the current Interacter implementation for wmf files uses the Windows API if a number of files are placed in one file and the combined file saved, the component files must be in a format other than wmf. :hardcopyfmt key Sets hardcopy output format for this graph only HP_GL => 1 HP-GL EPS => 2 PostScript RAST => 3 Raster Graphic PCX_BMP => 6 PCX/BMP LOTUS => 7 Lotus PIC DXF => 8 DXF CGM => 9 Computer Graphics Metafile WPM => 10 Windows Print Manager WMF => 11 Windows Meta File HP_GL2 => 12 HP_GL/2 *************************************************************** :cont option section Key words and arguments for :cont :toolbox Opens a windows to allow user input into graphics screen. The toolbox allows B34S users to interactively draw complex economics diagrams that can be moved into Word. The quickest way to get into this command interactively is through the 'Menu' Command in the Display Manager and select the DRAW command. :graphpvocab lists vocab of graphp :grarea array(4: x1 y1 x2 y2) :grunits array(4:xleft,ylower,xright,yupper) :grviewport array(4:x1 y1 x2 y2) Defines graphics viewport. Same as :grarea except current user units are recalculated to ensure the image size remains unchanged instead of being rescaled. Character size remains unchanged. :pgunitstogrunits x y gr_x gr_y x and y can be elements or an array. The variables gr_x and gr_y are automatically created. These names must be used to refer to these values due to the fact we are creating variables inside a parsed command. :pgunitsfromgrunits x y pg_x pg_y x and y can be elements or an array. The variables pg_x and pg_y are automatically created. These names must be used to refer to these values due to the fact we are creating variables inside a parsed command. :replayarea array(4: xx y1 x2 y2) Sets area for :replay to work :replay filename Loads a file into the graphics area where further processing can be done. Files loaded are HP-GL, HP-GL/2, GCM, Lotus PIC and WMF format. :grloadimage filename Loads BMP and PCX into graphics area. :grprintimage filename Dumps contents of graphics area to a file or a printer. :grplotmode key N => normal mode overwritting. O => OR plotting mode. A => AND plottong mode. E => EOR/XOR (Exclusive or) plotting mode. Example :grplotmode n :grarc array(5:xpos,ypos,radius,sangle,aangle) Draws a circular arc. xpos = x co-ordinate of circle centre ypos = y co-ordinate of circle centre radius = radius of circle in plotting units sangle = Arc start angle in degrees counter clockwise from 3 o'clock aangle = arc angle in degrees counter-clockwise Note: Arc can be filled with :grfillpattern :grarcrel array(3:radius,sangle,aangle) Draws a circular arc centered at current position. radius = radius of circle in plotting units. sangle = Arc start angle in degrees counter clockwise from 3 o'clock aangle = arc angle in degrees counter-clockwise. Note: Arc can be filled with :grfillpattern :grarrow array(4:xfrom,yfrom,xto,yto) itype Optional argument itype = 1 simple = 2 outline filled see grfillpattern :grarrowjoin array(:xtail ytail xhead yhead) itype Optional argument itype = 1 simple = 2 outline filled see grfillpattern :grblockcopy array(:xsour ysour xdest ydest width height) :grblockmove xsour ysour xdest ydest width height :grcircle array(:xpos ypos radius) Arguments can be arrays. If so pass three arrays. For futher information see grfillpattern. :grcirclerel radius :grellipse array(:xpos ypos radius ratio) Arguments can be 4 individual arrays for multiple ellipses. :grellipserel array(:radius ratio) :grlineto array(:xpos ypos) For arrays see grjoin :grlinetorel array(:dxpos dypos) :grmarkerrel marker Marker is an integer is range 1-20 Note: for character size adjustment of a dot then returns to standard size. :charsize array(:.5 .5) :grmarker gr_x gr_y 14 :charsize array(:1. 1.) :grparallel array(:xpos1,ypos1,xpos2,ypos2,apslen) itype apslen = length of axis parallel side itype = 1 y axis parallel = 2 x axis parallel :grparallelrel array(:dxpos,dypos,apslen) itype :grtrapezium array(:xpos1,ypos1,xpos2,ypos2, alen1 alen2) itype alen1=length of axis parallel side ending at xpos1 ypos1 alen2=length of axis parallel side ending at xpos2 ypos2 itype = 1 y axis parallel = 2 x axis parallel :grtranpeziumrel array(:dxpos,dypos,alen1,alen2) itype alen1 = length of axis-parallel side starting at current position alen2 = length of axis-parallel side ending at current position itype = 1 y axis parallel = 2 x axis parallel :grtrianglerel array(:dxpos2 dypos2 dxpos3,dypos3) :grcharlength string rlength calculates relative length of a string in rlength. With fixed spacing rlength=len(string). With porportional spacing these are not the same. rlength is a real*8 variable. :grcharspace ichr space Allows the proportional spacing table to be reset at runtime. Space for 'I' is .56 of its fixed space value. Since 'I' is code 73 the command :grcharspace 73 .45 resets 'I' smaller to .45 ichr = character code mist be 32-126 or 161-255 space = relative character space Note: ichr and space can be arrays ichr=0 => reset to defaults :grcharunderline key on => underline on off => underline off :grsaveimage fname use name.pcx or name.bmp Example :grsaveimage 'my.pcx' :grfileinfo fname info info(1) file type info(2) image width info(3) image height info(4) info(5) info(6) Info(1) codes: -1 File does not exist. 0 Unable to determine file type. 1 Windows .bmp 2 pcx format 3 Windows metafile 4 HP-GL plotter file 5 HP-GL/2 plotter file 6 Computer graphics Metafile cgm 7 Lotus PIC file 8 Acorn Draw 9 DEC LN03+Tektronix 4014 10 Postscript or EPS 11 HP PCL 12 Epson ESC/P2 13 Epson ESC/P 14 AutoCAD DXF See Interacter Documentation for further help. Example: :grfileinfo 'test.wmf' ii :grinputdevice key K => keyboard M => mouse D => digitising tablet :grinputlimits array(:xleft ylower xright yupper) :grdistanceline array(:x1 y1 x2 x2 xcheck ycheck) method rdist Defines a line and a check point and gets distance in rdist method nearest perpend :grinsidecircle array(:xpos ypos radius xcheck ycheck) isin Finds if point xcheck ycheck is in circle. If so isin=1. :grinsideellipse array(:xpos ypos radius ratio xcheck ycheck) isin Finds if point xcheck ycheck is in ellipse. If so isin=1. :grisidepolygon xpos ypos xcheck ycheck isin Finds if point xcheck ycheck is in polygone. Note that xpos and ypos are arrays. If so isin=1 :grintersectionline array(:x1 xy x2 y2 x3 y3 x4 y4) xinter yinter istatus xinter intersection points yinter intersection points istatus => 0 lines parallel and collinear => 1 lines parallel not collinear => 2 intersec outside => 3 intersect on line 1 => 4 intersect on line 2 => 5 intersect on both lines :grborder Draws a border around graph area. Not used in full screen mode. :pgarea array(4:x1 x2 y1 y2) Defines a relative position. x1, x2, y1, y2 are < 1. and > 0.0 :pgunits array(4:xmin ymin xmax ymax) Defines the units of x and y. :pause Stops processing until (cr). :pause clear Pause and wait for a key. If the optional key clear is present, the screen will be cleared after the next key it hit :grarerclear Clear graphics area. :pgborder Draws a border around presentation graph area. :grjoin array(:x) array(:y) Variants :grjoin array(:x1 y1 x2 y2) :grjoin x1 y1 x2 y2 :grjoin array(:x1) array(:y1) array(:x2) array(:y2) Note that array must have been built prior to call to command or with pgunitstogrunits. array(:x1 y1 x2 y2) can be used in place of x1 y1 x2 y2 if variables built prior to command. If multiple x1 values are passed, multiple lines are drawn. If either x or y are missing, point will be dropped. :grjoinrel array(4:x1 y1 dx1 dy1) Draws from a specified position to a new relative position. :grcurve array(:x) array(:y) nstep Draws a spline through a series of points. nstep is optional argument. default = 32 If either x or y are missing point will be dropped. :grmarker array(:x) array(:y) marker (1-20) Marker is optional If either x or y are missing, point will be dropped. :grpoint array(:x y) Set to point x, y. :grpointrel array(:dx dy) Set to point relative to current position. :grmoveto array(2:x1 y1) Move to point x1, y1. :grmovetorel array(2:dx1 dy1) Move relative to current point. :pspaceon Turns on proportional space. :pginfo Lists out graphics settings :pspaceoff Turns off proportional space. :charsize array(2:width height) Set character size. Example of making the dot smaller :charsize array(: .5 .5) :grmarker gr_x gr_y 14 :charsize array(:1. 1.) :charjustify key Sets how charout outputs. C => center L => left justified R => rightjustified :charout array(2:xpos ypos) string Draws in GR area. See also charoutrel to add text. :charoutrel string :charrotate r8 Measured counter clockwise from horizontal. :chardirection key h => horizontal v => vertical :charslant r8 r8 => Range -60. to + 60. :color key Color keywords black red yellow green cyan blue magenta white gray bred byellow bgreen bcyan bblue bmagenta bwhite :colorn i4 i4 in range (0-255). The command icolor(red) can be used to set the base color which can be adjusted. The :colorn command allows exact control over colors. black => 0 - 15 light red => 16 - 31 dark red => 32 - 47 light yellow => 48 - 63 dark yellow => 64 - 79 light green => 80 - 95 dark green => 96 - 111 light cyan => 112 - 127 dark cyan => 128 - 143 light blue => 144 - 159 dark blue => 160 - 175 light magenta => 176 - 191 dark magenta => 192 - 207 white => 208 - 223 light grey => 224 - 239 dark grey => 240 - 255 :grrectangle array(4:x1 xy x2 y2) Draws a rectangle. :grpolygonsimple array(:x) array(:y) Draws a simple polygon. Borders must not cross. If borders cross, use :grpolygoncomplex. :grpolygoncomplex array(:x) array(:y) Draws a complex polygon. Borders must not cross. If borgers do not cross, use :grpolygonsimple. :grpolygongrad array(:x) array(:y) ikey Draw irregular polygon using graduated color fill. ikey codes 1 => bottom-to-top 2 => left-to-right 3 => top-to-bottom 4 => right-to-left :grfillpattern index(istyle idense iangle) Sets fill pattern. Used with :grrectangle :grpolysimple :grpolycomplex :grpolydongrad istyle codes 0 outline 1 hatch -1 hatched no outline 2 cross hatch -2 cross hatch no outline 3 mixedcolor 4 solid idense codes 1 sparse 2 medium 3 dense1 4 dense2 5 dense3 iangle codes 1 diagonal sloping up 2 diagonal sloping down 3 fill horizontal lines 4 fill verticle lines :linetype key key codes solid (Default) dotted dashed dotdash dotdotdash (device dependent) longshort (device dependent) short (device dependent) :linewidth int array of 2 elements Sets line width in pixels for screen and hard copy. Default is :linewidth index(1 1) :grpolyline array(:x) array(:y) Draws a poly-line through a series of absolute coordinates. :grtriangle array(:x1 y1 x2 y2 x3 x4) Draws a triangle. :grrectanglerel arrray(:width height) Draws a rectangle ad correct point. :heading 'Heading here' key Heading can set up to 72 characters. Optional key set as L => left C => center R => right :grcharset filename Loads a character set file File names standard.chr General Purpose character set. standden.chr Danish variant of standard. standfra.chr French variant of standard. standger.chr German variant of standard. standita.chr Italian variant of standard. standnor.chr Norwegian variant of standard. standswe.chr Swedish variant of standard. standuk.chr UK variant of standard. simplexr.chr Similar to standard but with more detail. duplexr.chr More detailed that simplexr. triplexr.chr Heavier variant than duplexr. complexr.chr More tapered segments than triplexr. complexi.chr Italic version of Complex Roman. triplexi.chr Italic version of triplex Roman. simplexs.chr Handwritten style. complexs.chr More detailed variant of simplexs. simplexg.chr Greek characters added to simplexr. complexg.chr More detailed simplexg. gothicen.chr Very detailed old English style. gothicit.chrt Variant of gothicen. roman.chr Times Roman romanbld.chr Roman Bold swiss.chr Swiss / Helvetica style font swissbld.chr Swiss / Helvetica bold font Example: :grcharset 'roman.chr' :grcharfont ikey Sets Hardware fonts. ikey codes Fixed Proportional 1 Helvetica Courier 2 Helvetica ital Courier ital 3 Helvetica bold Courier bold 4 Helvetica bold/Ital Courier bold/ital 5 Times Roman Courier 6 Times Roman ital Courier ital 7 Times Roman bold Courier bold 8 Times Roman bold/ital Courier bold/ital :rxtick r8 Sets relative tick size. Default = 1.0 :rytick r8 Sets relative tick size. Default = 1.0 :rztick r8 Sets relative tick size. Default = 1.0 Note: For next commands key can be a character string or a string of letters. A character string, like that used in graph, is recommended. :xlabeltop ' ' key Sets text up to 90 and key where key must be L, C or R for left, centered, or right. :xlabeltop will override the :heading. Use :heading to change the size of the title. Use :xlabeltop to write near the top of the graph. :xlabel ' ' key Sets text up to 90 and key where key must be L, C or R for left, centered, or right. Key can be set as 'R' or R. :ylabelleft ' ' key Sets text up to 90 and key where key is a 2 level code. Key can be set as 'TV' or TV. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :ylabelright ' ' key Sets text up to 90 and key where key is a two level code. Key can be set as 'TV' or TV. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :zlabelleft ' ' key Sets text up to 90 and key where key is a 2 level code. Key can be set as 'TV' or TV. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :zlabelright ' ' key Sets text up to 90 and key where key is a two level code. Key ca be set as 'TV' or TV. Position 1 is: T -> Starting at top edge C -> Centered (default) B -> Ending at bottom edge Position 2 is: V -> Verticle (default) R -> Rotated 270 degrees 9 -> Rotated 90 degrees :xlabelpos r8 Sets relative position of xlabel. r8 must be in range 0 to 1.0. Default = .7. Smaller numbers mean nearer to figure. :ylabelpos r8 Sets relative position of ylabel. r8 must be in range 0 to 1.0. Default = .8. Smaller numbers mean nearer to figure. :zlabelpos r8 Sets relative position of zlabel. r8 must be in range 0 to 1.0 Default = .8. Smaller numbers mean nearer to figure. :xscale real array Sets user x label values. Length of array must be le 100. :xscale array(:4 8 12) Note: pgxscale controls display. :yscale real array Sets user y label values. Length of array must be le 100 :yscale array(:4 8 12) Note: pgyscale controls display. :zscale real array Sets user z label values. Length of array must be le 100. :zscale array(:4 8 12) :grpalettehls index(ncolor ihue ilight isat) Sets colors using hue light and saturation. ncolor in range 0-255 ihue in range 0-360 0 => blue 60 => magenta 120 => red 180 => yellow 240 => green 300 => cyan ilight in range 0 = 100 0 => black 100 => white isatur in range 0 to 100 0 => gray 100 => pure color :grpaletteinit Restores default settings. :grpalettergb index(ncolor ired igreen iblue) Controls colors by % of red green blue ncolor in range 0-255 ired in range 0-255 igreen in range 0-255 iblue in range 0-255 *********************************************************** Higher level commands requiring detailled access to Interacter manuals. :pgnewplot index(itype nsets layout ireset) index(nvalue1 nvalue2 nvalue3) nvalue = 1 element integer array usually = 2 element array for contour surface = 3 element array for volume plots :pgnewgraph index(nsets nvalue1 nvalue2) array(:cuml layout grtype) nsets => Number of datasets nvalue = 1 element integer array usually = 2 element array for contour surface Note: elenets 1 # of points cum1 = 'c' array containing c for cumulative, ' ' otherwise layout 3 => 3 dimensional A => adjacent bars in histograms/bar or anti clockwise wedges B => View 3-d from back S => Fit spline.. display spider tags V => Variable grid size 3D surface plots Variable length spider tags Value labels on 2D non-cumulative bar-charts histograms H => Height dependent contour colors on 3D surfaces or high/low histogram plot. F => Fill-based 2D contour plot or plot bars in front of each other on 2D cumulative histograms C => Point-dependent colors on 3D scatter plots T => Tile-dependent colors on 3D contour plots. grtype B => bar C => contour F => function plot H => histogray L => line plot P => pie chart S => scatter T => table X -> x/y co-ordinate plot :pgxscale string 'T' => ticks outside 'I' => ticks inside 'N' => numbering :pgyscale string 'T' => ticks outside 'I' => ticks inside 'N' => numbering :pgxscalepos r8 :pgyscalepos r8 :pgyscaleangle array(2:tangle,sangle) tangle => Y axis tick mark angle sangle => Y axis scale value string angle in degrees counter-clockwise from horizontal :pgyticklength r8 r8 => relative length of Y axis tick mark default=1.0 :pgytickpos array(2:xleft,xright) xleft => x position of left Y axis tick mark xright => x position of right Y axis tick mark xright restore default :pgaxes Draws pg axis :pgaxesxy array(2:x1 y1) Sets position of axes. Default is 0.0 0.0 :pgxgraticules key Key can be solid (Default) dotted dashed dotdash dotdotdash longshort short :pgygraticules key Key can be solid (Default) dotted dashed dotdash dotdotdash longshort short :pgzgraticules key Key can be solid (Default) dotted dashed dotdash dotdotdash longshort short :pgxscaletop key Places x scale on top. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgyscaleleft key Places y scale on left. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgyscaleright key Places y scale on right. Key is a 2 level code t -> places ticks outside I -> places ticks inside N -> Numbers axis :pgstyle index(iset istyle1 istyle2 istyle3 icol1 icol2) iset = Data set number istyle1= Plot type specific style # 1 istyle2= Plot type specific style # 2 istyle3= Plot type specific style # 3 icol1 = Plot type specific color # 1 icol1 = Plot type specific color # 2 For further information see Interacter Manual :pglineplot array(:x) :pgcliprectangle key G => main graphics area P => PG area :pgconfill2granul igan igan ge 1 sets fill granularity :pgcontourlabel iset label iset contour number label label (max 10 characters) :pgdecimalplaces ndec ndec => number of decimal places < 0 => auto select :pgelevation angle 0. le angle le 45. :pggriddirection igrid 3D surface plots 3 => both x and y 2 => perpendicular to y 1 => perpendicular to x 0 => no grid :pggridlines igrid 3D contour 1 => height dependent grid lines 2 => no grid lines 3 => grid lines drawn in background color :pgmarker iset marker iset => data set marker => marker code 0-9, 1-52 Note: This option must be used with style2 or pgstyle. For further detail see Interacter documentation. ipgstyle Market Type Marker 1 Digits 0-9 2 Letters 1-26 (A-Z) 27-52 (a-z) 3 Markers 1-20 as drawn by igrmarker 4 Symbols 33-126 or 161-255 See symbols.chr :pgmarketfrequency istart ievery Sets marker frequency and start :pgrotation angle Angle rotation for 3D plot views :pgscalling xscalkey yscalkey key LIN LOG :pgstyle3daxes index(istyle icol11 icol12 icol13 icol21 icol22 icol23) istype 0 => outline 3 => mixed 4 => solid icol1 & icol2 set primary and secondary fill colors. Color codes can be set using icolor(' ') command :pgstyleoutline icol Sets outline color :pgunitspolar rmax Maximim radius for polar plots :pgunitsz array(:zmin zmax) :pgxkeypos relpos Sets relative key position. 0. le relpos le 1. :pgxscaleangle array(:tangle sangle) tangle= axis tick mark angle sangle= axis scale value string angle :pgxtickpos array(:ybottom,ytop) :pgxuserscale spoint :pgxuserscalehist ibars ibars is an integer array :pgylabelpoc r8 r8 is a relative position :pgyuserscale spoint spoint is an array of user scales. max # = 100 No argument disables. :pgyuserscalebar ibars ibars = array of histogram scales No argument disables. :pgzscaleangle array(:tangle sangle) tangle= axis tick mark angle sangle= axis scale value string angle :pgkeyall descr layout descr = array of discriptions layout V 9 X B R E => end of lines P Set both arguments as character*8 :pgkeyall namelist(income price) '9B' For further detail see Interacter Documentation :pgkeysingle iset xpos ypos descr Sets a key. Can be supplied for a number of points. :pgxtext descr x axis dscriptions. Pass as character*8 :pgxtexttop descr x axis dscriptions. Pass as character*8 :pgytextleft descr Labels left Y axis. Pass as character*8 :pgytextright descr Labels right y axis. Pass as character*8. :pgzscale key I => inside T => outside N => no ticks :pgztext descr Labels z axis :pgbarchart xvalues :pgerrorbars ylow yhigh For line plot Note: Must supply right # :pghighlow ylow yhigh For histogram Note: Must supply right amount :pghistogram yvalue :pgpiechart pival sangle explode pival => array of values to be plotted sangle => start angle explode => character*1 array with blanks or E to explode. If argument left off assumes blanks :pgscatterplot xvalue yvalue :pgscatterplot3d xvalue yvalue zvalue :pgscatterplot3dcol xvalue yvalue zvalue colors Same as pgscatterplot3d except supply colors. :pgtableinteger ivalues ivalues => array of integer values :pgtablereal rvalues fmt rvalues => array of real values to be plotted in table. fmt is optional. Default g16.8. :pgxypairs xvalue yvalue :pgxyztriplets xvalue yvalue zvalue :pgconfill2irreg zvalue zcontr xgrid ygrid zvalue => is nxdim nydim zcontr => sets nc set on pgnewplot xgrid => is nxdim ygrid => is nydim :pgconfill2reg zvalue zcontr zvalue => is nxdim nydim zcontr => sets nc on pgnewplot :pgcontour2irreg zvalue zcontr xgrid ygrid zvalue => is nxdim nydim zcontr => sets nc set on pgnewplot xgrid => is nxdim ygrid => is nydim :pgcontour2reg zvalue zcontr zvalue => sets nxdim nydim zcontr => sets nc on pgnewplot :pgsurf3data zvalue zvalue => is nxdim nydim :pgsurf3datacol zvalue icol zvalue => is nxdim nydim icol => is nxdim nydim :pgsurf3datacont zvalue zcontr zvalue => is nxdim nydim zcontr => is nc on pgnewplot :pgsurf3step zvalue zvalue => is nxdim nydim :pgsurf3stepcol zvalue icol zvalue => is nxdim nydim icol => is nxdim nydim :pgsurf3stepcont zvalue zcontr zvalue => is nxdim nydim zcontr => is nc on pgnewplot :pgvolume3col icolr index(nxdim nydim nzdim) icolr => is nxdim nydim nzdim :pgvolume3cont value index(nxdim nydim nzdim) contr contr => nc array of contour values :pgxyzsearchbox array(:boxwidth boxhgt) :pgxyztogrid x y z zrec index(n1 n2) x => array of x data y => array of y data z => array of heights zrec is n1 by n2 and contains a matrix that can be plotted in 3D. Note: zrec is real*8 :pgjoin2 array(:xpg1 ypg1 xpg2 ypg2) :pgjoin3 array(:xpg1 ypg1 zpg1 xpg1 ypg2 zpg2) :pgpolygoncomplex2 xpg ypg Arrays of x and y points to draw # of points LE 4095 :pgpolygoncomplex3 xpg ypg zpg Will draw a 3D figure. # of points LE 4095 :pgpolyline2 xpg ypg Arrays of x and y points to draw # of points LE 4095 :pgpolyline3 xpg ypg zpg Will draw a 3D figure # of points LE 4095 :pgunitxfromgrunitsp gxpos gypos angle radius Can be supplied as arrays :pgunitstogrunits3 pgxpos pgypos pgzpos gxpos gypos :pgunitstogrunitsp angle radius gxpos gypos *********************************************************** :final option section call graphp(:final); Terminates the progessing of the graph and either displays or produces hardcopy depending on settings on the :start option cl. Functions useful for graphp command: infograph - Obtain Interacter Graphics INFO r=infograph(n); n in range 1-14 1 => Current x plotting position 2 => Current y plotting position 3 => Current character width 4 => Current character height 5 => Mouse x position 6 => Mouse y position 7 => Left limit on graphics area 8 => lower limit on graphics area 9 => Right limit on main graphics area 10 => Upper limit on main graphics area 11 => Lower x co-ordinate limit 12 => Lower y co-ordinate limit 13 => Upper x co-ordinate limit 14 => Upper y co-ordinate limit r is real*4 Note: This routine must be used on distinct call graphp(:cont) calls to be updated properly. This routine has no use outside graphp. Example of graphs with call graph and call graphp. b34sexec matrix; call loaddata; call graph(cac :plottype hist2d /$ :heading 'CAC Ratio ordered by Size of Firm' :nolabel :nokey :colors black black :pspaceon :file 'CACPLOT.WMF' :xlabel 'Participants arrayed by emissions size' :ylabelleft 'CAC ratio' 'C9' ); /$ graphp implementation icolor=223; call graphp(:start /$ :file 'newfig2.wmf' /$ :hardcopyfmt wmf ); call graphp(:cont :graphpvocab); call graphp(:cont :grarea array(:0. 0. 1. 1.) :grunits array(:1. 0. 168. 3.) :pgarea array(:.1 .1 .9 .9) :pgunits array(:1. 0. 168. 3. ) :color black :pgborder :pspaceon :pgxscale 'N' :pgyscaleleft 'tN' :xlabel 'Participants arrayed by emissions size' :ylabelleft 'CAC ratio' C9 /$ :pgnewplot index(1,1,0,1) index(norows(cac)) :pgnewgraph index(1 norows(cac) 0) array(:' ',' ','H') :pgstyle index(1,-4,3,1,icolor,icolor) :pghistogram cac /$ :pgnewplot index(4,1,0,1) index(norows(cac)) :pgnewgraph index(1 norows(cac) 0) array(:' ',' ','L') :pgstyle index(1, 0,0,0,icolor,icolor) :pglineplot constant /$ :toolbox ); call graphp(:final); b34srun; Example of a distribution plot with a user axis b34sexec matrix; call echooff; call getsca('c:\b34slm\findat01.mad' :mad :member D_AA); YMean=Mean(D_AA); YSigma2=Variance(D_AA-YMean); call garchest(res1, res2, D_AA,func,1,nbad :cparms array(:YMean, YSigma2) :garorder idint(array(:1)) :gmaorder idint(array(:1)) :print ); _sqrmat=array(dmax1(norows(res1),norows(res2)),2:); _sqrmat(,1)=res1; _sqrmat(,2)=res2; _sqrmat =goodrow(_sqrmat); res1=_sqrmat(,1); res2=_sqrmat(,2); Residual=goodrow(res1); Sigma=goodrow(dsqrt(goodrow(res2))); et=Residual/Sigma; x=et; /; data in variable x ibars=13 ; /; Automatic calculation not used. /; call datafreq(x _table :equal ibars midpts); /; midpoints set as -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 upper= 5.5; lower=-5.5; call datafreq(x _table :equaluser ibars midpts lower upper); /; call tabulate(_table midpts); /; a test case showing defaults call character(cc,'Default plot '); call graph(_table :plottype hist2d :heading cc :pspaceon :pgyscaleright 'i' :pgborder :pgxscaletop 'i' :colors black bblue bred ); /; +++++++++++++++++++++++++++++++++++++++++++++++++++++ /; we set range of x axis and y axis. /; By defining xmax2 etc it allows a fudge as has been /; done with ymax2 /; Note that datafreq gives us the exact midpoints of each /; rectangle xmax=dmax(x); xmin=dmin(x); /; xmax2=xmax; /; xmin2=xmin; xmax2= 6.5; xmin2=-6.5; ymin2=0.0; ymax2=dmax(_table)+(dmax(_table)/20.); uscale=midpts; /; Make sure no rectangles are 0.0 height. Add a "fudge" /; testing xmin=lower; do ii=1,norows(_table); if(_table(ii).le.0.0)_table(ii)=.1e-3; enddo; call graphp(:start :file '_table.wmf'); call graphp(:cont :grarea array(:0.0 0.0 1. 1.) :grunits array(: 1. xmin2 xmax2 dfloat(norows(_table))) :pgarea array(: .1 .1 .9 .9) :pgunits array(: xmin2 ymin2 xmax2 ymax2) :pgborder :pspaceon :xscale uscale :pgxscale 'TN' :pgyscaleleft 'n' :xlabel 'Distribution of Standardized Residual' :ylabelleft '# of Cases' /; :ylabelleft '# of Cases' 'Cr' /; :ylabelleft '# of Cases' 'C9' :heading 'This is a test histogram' :pgnewgraph index(1,norows(_table)) array(:' ' ' ' 'H') :pgstyle index(1,4,0,0,160,20) :pghistogram _table ); call graphp(:final); /; Quick see what we have!! call grreplay('_table.wmf'); b34srun; Error Messages from Interacter: 1 Error opening file. 2 Error reading or writting to a device. 3 Error closing a file. 4 Number too large iun string to numeric conversion. 5 Graphics not supported on requested printer. 6 Screen mode not supported for load/save operation. 7 Max number of windows exceeded. 8 Window buffer space exceeded. 9 Invalid window co-ordinates. 10 No substring found. 11 More than one decimal point in number. 12 Invalid character detected. 13 Operating system command error in an OS routine. 14 Invalid text co-ordinates for clear operation. 15 Centred string truncated (exceeds screen or window width). 16 X or Y graphics unit range is invalid. Default of 0-1 used. 17 Window destination partly or wholly off-screen. Destination co-ordinates adjusted. 18 Numeric to string conversion error. String filled with *'s. 19 All options start with '-' in a menu. 20 Radius of a circle/ellipse, or height ratio of an ellipse is =< zero or an arc angle = 0. Radius and ratio must be positive, arc angles must be non-zero. 21 The length of the axis-parallel side of a parallelogram is =< zero 22 A rectangle/triangle/parallelogram has been specified with either no width or no height. 23 No program name specified to IOsExecute. 24 Too many data values specified to IPgNewGraph or IPgNewPlot for a cumulative plot. Reset to maximum internal limit. 25 Too many bytes in character set definition during conversion from ASCII to binary file format. 26 Not used in this release. 27 Incompatible or unsupported image file format 28 Borders cross in IGrPolygonSimple. Unable to fill. 29 Blank field entered in a numeric input routine. 30 All fields protected on a form. 31 Field types do not match in Forms Manager. 32 Field value is undefined in Forms Manager. 33 Internal character data storage area full in Forms Manager. 34 Requested page size too large for raster image buffer. (Generated by raster graphics hardcopy driver.) 35 No error of this type in currect release 36 Attempt to store an out of range form field value. 37 Non-fatal error in INTERACTER Form Definition File. 38 Not an INTERACTER Form Definition file. 39 Failed to select requested graphics colour. Current colour unchanged. 40 Description Number of files matched by IOsDirInfo/List exceeds size of supplied array(s). 41 Unknown or unsupported screen mode requested in call to IScreenModeN. 42 Unknown colour name or number specified to IGrColouror IGrColourN. Current graphics colour remains unchanged. 43 Contour heights do not increase monotonically in contour/surface routine. 44 Invalid X and/or Y range specified to IGrArea, IGrViewportor IGrReplayArea. Range reset to 0-1. 45 Null menu in IdGrHardcopyDriveror IdScreenMode. 46 No graphics hardcopy driver currently selected, in IdGrHardcopyOptions. 47 Not used in this release. 48 Invalid dataset number in IdPgStyles. 49 Fill too complex in IGrPolygonComplex. 50 Form contains no [window] specification in IFormOpenWindow. 51 A search box contains too many points in IPgXYZToGrid. 52 Invalid number of buttons specified to IdMessage. 53 Unable to find software font file in IGrCharOut to substitute for unavailable hardware font. 54 Mismatch between driver/device number in hardcopy options file and current selections in IGrHardCopyOptLoad. 55 Source and target file names are the same in a copy command in IOsCopyFile. 56 Text buffer too small in IWinEditFile. 57 Attempt to create a radio button group for which there are insufficent check-box fields available 58 A min value is larger than a max value in IPgHighLow. A zero height bar will be drawn. 59 Too many grid columns requested in IGridDefine. 60 Invalid column type requested in IGridDefine. 61 No window open for grid in IGridShowor IGridEdit. 62 Invalid grid starting position in IGridShowor IGridEdit. 63 No Windows printer available in IdGrHardcopyOptions. 64 Bit image printer dump failed under Windows 65 Return buffer too small in IOsVariable 66 Invalid character code specified to IGrCharSpace The following Exit Codes occur in the event of INTERACTER detecting an unrecoverable error. In this case interacter, calls the IOsExitProgram routine and passes an exit code to the operating system. The following is a list of the exit codes, which routines generate them and the likely cause. If these errors occur please describe the circumstances and report to the b34s developer at hhstokes@uic.edu. Codes 1 to 20 are reserved for use by INTERACTER. 1 IScreenModeNA DOS screen mode which was expected to be available was not actually selected. This is usually due to incorrectly identifying the DOS display type. 2 IOsExecute INTERACTER was unable to execute the requested program for some reason, possibly because the name was incorrectly specified or does not exist within the current execution path. 3 Not used 4 IOsExecuteProgram chaining is not supported under most 32-bit DOS protected mode compilers. 5 Not used. 6/7 IScreenOpen INTERACTER was unable to get the Unix IdDisplay terminal driver characteristics 8 Non used 9 IScreenOpen The X Windows display type was requested but IdDisplay the Xlib call to open the X display failed. IDisplay Display type 453 can only be used under an X Windows server. 10 IScreenOpen An X Windows font name specified using the IdDisplay TEXTFONT initialisation file keyword was not Idisplay found. 11/12 IScreenOpen A standard X Windows font which IdDisplay INTERACTER expected to be available (e.g. IDisplay '6x13') was not found. Make sure you have the IGrCharOut X11R4/5/6 fixed width fonts available. IGrCharOutRel 13 IScreenOpen An attempt has been made to select a graphics IScreenMode screen mode when the currently identified IGrInit display type does not support graphics. Check the selected INTERACTER display type. 14 IGrSymbConvertOld style 25-piece symbol sets are no longer supported. Use IGrCharConvertto create a character set instead. GRCHARSET - Set Character Set for Graphics call grcharset(' '); Sets the character set for graphics. If user has access to Interacter Documentation, custom characters can be built and the charconv command under OPTIONS can be used to make a user character set. Under the Display Manager the character set can be interactively set. The command :markpoint 1 1 2 12 access the user character 12 if call grcharset had been given. Alternatively the character set can be set in the interact.ini file. As presently setup full control of character sets is only available in call graphp. Default graphics initialization overrides grcharset settings in usual graph command. Usually users do not have to use this command. GRREPLAY - Graph replay and reformat command. call grreplay('file name'); Will display a graph that was saved using the :file option in the call graph( ); command. Advanced options in the grreplay command allow: - Reformatting graph save files. - Combining a number of graph files into one file. - Zooming sections of a graph save file. :file ' ' Saves the graph in a file. If file is present, the graph will not show on the screen. If a blank string of the form ' ' is passed and B34S is running on windows, the file will be placed on the clip board as long as the file save type is wmf. Due to the fact that the current Interacter implementation for wmf files uses the Windows API if a number of files are placed in one file and the combined file saved, the component files must be in a format other than wmf. For example the following code makes a combined graph file that consists of graphs of the series gasin and gasout. This file is saved, viewed and placed on the clipboard. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call graph(gasout :file 'p1.hp1' :hardcopyfmt HP_GL); call graph(gasin :file 'p2.hp1' :hardcopyfmt HP_GL); /$ view the two files call grreplay('p1.hp1','p2.hp1'); /$ save the two files p1.hp1 and p2.hp1 in new.wmf call grreplay('p1.hp1','p2.hp1' :file 'new.wmf' :hardcopyfmt wmf); /$ view the new combined file call grreplay('new.wmf'); /$ Place the two files on the clip board call grreplay('p1.hp1','p2.hp1' :file ' ' :hardcopyfmt wmf); b34srun; :hardcopyfmt key Sets hardcopy output format for this graph only. HP_GL => 1 HP-GL EPS => 2 PostScript RAST => 3 Raster Graphic PCX_BMP => 6 PCX/BMP LOTUS => 7 Lotus PIC DXF => 8 DXF CGM => 9 Computer Graphics Metafile WPM => 10 Windows Print Manager WMF => 11 Windows Meta File HP_GL2 => 12 HP_GL/2 Example: call grreplay('myplot.wmf' :hardcopyfmt PCX_BMP :file 'myplot.pcx'); reformats the graph. More than one file can be displayed as long as the number of files is 1, 2, 4 or 9. The command call grreplay('plot1.wmf','plot2.wmf'); displays one on top of the other. call grreplay('plot1.wmf', 'plot2.mmf' :file 'newplot.wmf'); call grreplay('newplot.wmf'); Combines a two plots into one plot and displays the combined plot. If 4 or 9 files are supplied, then, these are shown in a predefined form: twograph 1 2 fourgraph 1 2 3 4 ninegraph 1 2 3 4 5 6 7 8 9 threegraphv 1 2 3 threegraphh 1 2 3 Advanced options: The GRREPLAY command can be given a number of times to build a custom plot. In this mode or operation for the first call use the key :start If :start is given the only allowed options are :hardcopyfmt and :file. For all subsequent calls except the final call use the key :cont filename If :cont is supplied only the options :gformat, :area or :zoom are allowed For the final call use call grreplay(:final); Discussion: For each call one file is passed. The positioning of the file is controlled by either :gformat key i4 key is set onegraph twograph threegraphh threegraphv fourgraph ninegraph i4 is set to the graph number. Example where we have 3 graphs and want to display them in a four way graph. call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat fourgraph 1); call grreplay(:cont 'plot2.wmf' :gformat fourgraph 2); call grreplay(:cont 'plot3.wmf' :gformat fourgraph 3); call grreplay(:final); to show the combined graph. As an alternative to the :gformat the option :area r8 array of 4 elements can be used. The elements are: 1. => x_left 2. => y_lower 3. => x_right 4. => y_upper Example call grreplay(:start); call grreplay(:cont 'plot1.wmf' :area array(:.0 .0 1. .5)); call grreplay(:cont 'plot2.wmf' :area array(:.0 .5 1. 1.)); call grreplay(:final); The Interacter graphics routines use Windows API calls to display WMF files which cannot be zoomed. If a file is NOT a WMF file the option :zoom r8 array of 4 elements Can be used to select just that portion of the source file to display. The elements are: 1. => x_left 2. => y_lower 3. => x_right 4. => y_upper Example of a zoom of plot1.cgm: call grreplay(:start); call grreplay(:cont 'plot1.cgm' :zoom array(:.5 .5 1. 1.) :area array(:.0 .0 1. .5)); call grreplay(:cont 'plot2.cgm' :area array(:.0 .5 1. 1.)); call grreplay(:final); Example to save a combined plot. call grreplay(:start); call grreplay(:cont 'plot1.wmf' :area array(:.0 .0 1. .5)); call grreplay(:cont 'plot2.wmf' :area array(:.0 .5 1. 1.)); call grreplay(:final); or call grreplay('plot1.wmf' 'plot2.wmf' :file 'newplot.wmf'); Comprehensive example showing building plots of the form 1 2 3 b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call graph(gasout :file 'p1.hp1' :heading 'Gasout hp_GL' :noshow :hardcopyfmt HP_GL); call graph(gasin :file 'p2.hp1' :heading 'Gasin HP_GL' :noshow :hardcopyfmt HP_GL); call grreplay(:start); call grreplay(:cont 'p1.hp1' :gformat twograph 1); call grreplay(:cont 'p2.hp1' :gformat twograph 2); call grreplay(:final); b34srun; b34sexec options ginclude('b34sdata.mac') member(res72); b34srun; b34sexec matrix; call loaddata; call graph(lnq :heading 'Ln Q' :file 'plot1.wmf' :noshow); call graph(lnl :heading 'Ln L' :file 'plot2.wmf' :noshow); call graph(lnk :heading 'Ln k' :file 'plot3.wmf' :noshow); call graph(lnrm1 :heading 'Ln rm1' :file 'plot4.wmf' :noshow); call graph(lnrm2 :heading 'Ln rm2' :file 'plot5.wmf' :noshow); call graph(P :heading 'P ' :file 'plot6.wmf' :noshow); call graph(m1 :heading 'M1 ' :file 'plot7.wmf' :noshow); call graph(m2 :heading 'M2 ' :file 'plot8.wmf' :noshow); call graph(L :heading 'L ' :file 'plot9.wmf' :noshow); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat onegraph 1); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat twograph 1); call grreplay(:cont 'plot2.wmf' :gformat twograph 2); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat fourgraph 1); call grreplay(:cont 'plot2.wmf' :gformat fourgraph 2); call grreplay(:cont 'plot3.wmf' :gformat fourgraph 3); call grreplay(:cont 'plot4.wmf' :gformat fourgraph 4); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat ninegraph 1); call grreplay(:cont 'plot2.wmf' :gformat ninegraph 2); call grreplay(:cont 'plot3.wmf' :gformat ninegraph 3); call grreplay(:cont 'plot4.wmf' :gformat twograph 2); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat ninegraph 1); call grreplay(:cont 'plot2.wmf' :gformat ninegraph 2); call grreplay(:cont 'plot3.wmf' :gformat ninegraph 3); call grreplay(:cont 'plot4.wmf' :gformat ninegraph 4); call grreplay(:cont 'plot5.wmf' :gformat ninegraph 5); call grreplay(:cont 'plot6.wmf' :gformat ninegraph 6); call grreplay(:cont 'plot7.wmf' :gformat ninegraph 7); call grreplay(:cont 'plot8.wmf' :gformat ninegraph 8); call grreplay(:cont 'plot9.wmf' :gformat ninegraph 9); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :gformat onegraph 1 :zoom array(:.33333 .33333 .66666 .66666)); call grreplay(:final); call grreplay(:start); call grreplay(:cont 'plot1.wmf' :area array(:.33333 .33333 .66666 .66666) :zoom array(:.33333 .33333 .66666 .66666)); call grreplay(:final); b34srun; GTEST Tests Output from a ARCH/GARCH Model call gtest(res1,res2,y,nacf); Tests the first and second moments of a ARCH / GARCH model subroutine gtest(res1,res2,y,nacf); /; /; res1 => First Moment Residual /; res2 => Second Moment Residual /; y => Input Series /; nacf => Number acf terms /; /; Plots made: /; /; acfa.wmf => acf of residual Moment 1 /; acfb.wmf => acf of residual Moment 2 /; acfy.wmf => acf of y series /; mqa.wmf => Q stats residual Moment 1 /; mqb.wmf => Q stats residual Moment 2 /; pacfa.wmf => pacf of residual Moment 1 /; pacfb.wmf => pacf of residual Moment 2 /; pacfy.wmf => pacf of y series /; resa.wmf => Plot of residual Moment 1 /; resb.wmf => Plot of residual Moment 1 /; Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix ; call loaddata; call load(gtest); arch=array(norows(gasout):); call olsq(gasout gasout{1} gasout{2} :print); call print('RESVAR',%resvar :); call garchest(res,arch,gasout,func,2,n :cparms array(2:%coef(3), %resvar) :nar 2 :arparms array(2: %coef(1) %coef(2)) :ngar 1 :ngma 1 :gmaparms array(:.05) :print ); call gtest(res,arch,gasout,48); b34srun; GWRITE Save Objects in GAUSS Format using one file call gwrite(x,xxname,unit); Saves object x on unit using name in xxname. The number of elements in x must be LE 1000 due to internal GAUSS limits on the size of a sentence. If the object to be passed is larger than 1000, use gwrite2. x - Object name xxname - Name in file unit - Fortran I/O unit Real*8 and Integer*4 objects supported. Note: Since GWRITE is a subroutine and must be loaded prior to use. Example: b34sexec matrix; call load(gwrite); call open(70,'testdata'); y=array(2,2:1 2 3 4); nn=namelist(y); call gwrite(y,nn,70); xx=rn(matrix(5,5:)); nn=namelist(xx); call gwrite(xx,nn,70); i=integers(1,23); ii=namelist(i); call gwrite(i,ii,70); call close(70); b34srun; Example 2 - Runs OLS in GAUSS b34sexec matrix; call load(gwrite); call open(70,'testdata'); x1=rn(array(100:)); nn=namelist(x1); call gwrite(x1,nn,70); yy=10. + x1 + rn(x1); nn=namelist(yy); call gwrite(yy,nn,70); call olsq(yy x1 :print); call character(cc,'ols("",yy,x1);'); call write(cc,70); call close(70); /$ run the file call unix('gaussb testdata > jj.out'); b34srun; /$ b34sexec options npageout writeout('Output from GAUSS',' ',' ') copyfout('jj.out'); b34srun; GWRITE2 Pass Data to Gauss in two files call gwrite2(x,xxname,unit); Saves object x on unit using name in xxname in GAUSS format. This command is used if object is larger than 1000. x => Object name xxname => Name in file unit => Fortran I/O unit Real*8 and Integer*4 objects supported. Note: Since GWRITE2 is a subroutine it must be loaded prior to use. gwrite2 makes a file xxname.fmt for each series. Hence if the object to be moved is < 1000 it may pay to use gwrite which has only one file. Example: b34sexec matrix; call load(gwrite2); call open(70,'testdata'); x1=rn(array(10000:)); nn=namelist(x1); call gwrite2(x1,nn,70); yy=10. + x1 + 10.*rn(x1); nn=namelist(yy); call gwrite2(yy,nn,70); call olsq(yy x1 :print); /$ /$ Do an OLS Model in GAUSS /$ call character(cc,'ols("",yy,x1);'); call write(cc,70); call close(70); call unix('gaussb testdata > jj.out'); b34srun; b34sexec options npageout writeout('Output from GAUSS',' ',' ') copyfout('jj.out'); b34srun; HEADER Turn on header call header; Turns on page numbering inside matrix command. Since a new page number is forced every time this command is found, it can be given multiple times inside the same job. HEXTOCH Convert a hex value to its character representation call hexttch(hex,ch); Converts a hex value to character. hex => character*1 character matrix of size 2*n ch => character*1 character vector of size n Extended Example b34sexec matrix; /$ Looking at Printable Characters ; i=integers(33,127); call igetchari(i,cc); call names(all); call tabulate(i,cc); call igetichar(cc,iitest); call chtohex(cc,hexcc); /$ Repack character*2 array save as character*1; /$ Next two statments work the same /$ hexcc2= array(norows(hexcc)/2,2:hexcc); hexcc2=c1array(norows(hexcc)/2,2:hexcc); hex1=hexcc2(,1); hex2=hexcc2(,2); call hextoch(hexcc,cctest); xx=transpose(hexcc2); call print(xx,hexcc2); call hextoch(xx,cctest2); call names(all); /$ get hexcc2 in a printable variable; blank=c1array(norows(hex1):); call names(all); c8var=catcol(hex1, hex2,blank,blank, blank, blank,blank,blank); call names(all); /$ call print(c8var); c8var=c8array(norows(c8var):transpose(c8var)); call tabulate(i,cc,iitest,hex1,hex2, cctest,cctest2,c8var); b34srun; HINICH82 Hinich 1982 Nonlinearity Test. call hinich82(x,m,g,l) Calculate Hinich(82) test for series x. x = input series. Must be set. m = Number of terms to average. Only set if :setm is in effect. g = Gaussianity test. Output by routine. l = linearity test. Output by routine. The test is performed over admissable range. See BTIDEN command for more detail. Using default settings, last two G and L values are mean and variance of prior G and L values. M is given the values -99. and -999 for these observations. Options: :meanonly - averages G & L only here m set as -99 variance where m -999 :setm - # of terms set in M. :smoothspec - smooth spectrum needed if x not white noise. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call hinich82(gasout,m,g,l:meanonly); call print('Mean Data for Hinich(82) Test on Gasout',g,l); m=17; call hinich82(gasout,m,g,l:setm); call print( 'Mean Data for Hinich(82) Test on Gasout M Set',m,g,l); call hinich82(gasout,m,g,l); call print('Hinich(82) Test on Gasout not Smoothed'); call tabulate(m,g,l); call hinich82(gasout,m,g,l:meanonly :smoothspec); call print('Mean Data for Hinich(82) Test on Gasout',g,l); m=16; call hinich82(gasout,m,g,l:setm :smoothspec); call print( 'Mean Data for Hinich(82) Test on Gasout Mean Set',g,l); call hinich82(gasout,m,g,l :smoothspec); call print('Hinich(82) Test on Gasout Smoothed'); call tabulate(m,g,l); b34srun; HINICH96 Hinich 1996 Nonlinearity Test. call hinich96(x,c,v,h) Calculates Hinich(96) v and h test for x. x = series c = sets number of lags. If c le 0, c defaults to .4. # of lags = nob**c. For detail on this test see Stokes (1997). Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call echooff; call hinich96(gasout,0.0,V,H); call print( 'Mean Data for Hinich(96) Test on Gasout',V,H); c=grid(.2 .45,.02); v=array(norows(c):); h=array(norows(c):); do i=1,norows(c); call hinich96(gasout,c(i),vv,hh); v(i)=vv; h(i)=hh; enddo; call print( 'Hinich(96) Test on Gasout for various c values'); call tabulate(c,v,h); b34srun; HPFILTER Hodrick-Prescott Filter. call hpfilter(data,datat,datadev,lamda); Uses Hodrick-Prescott filter to decompose data into trend (datat) and deviations from trend (datadev). Data must be real*8. If data is a matrix or 2d array, each column is transformed. data = real*8 series (can be a matrix) or 2d array). datat = trend part of series datadev = deviation part of series Lamda = sets the cost of incorporating fluctuations into the trend. Default = 1600. Prescott suggests 1600. for quarterly data. For yearly data set 1600 / 4**2 = 100. For monthly data set 1600 * 3**2 = 14,400. This command uses Prescott's subroutine that selects u(t) such that (1/T)sum((y(t)-u(t)**2)-(lamda/T)* sum((u(t+1)-u(t)-(u(t)-u(t-1)))**2 is minimized. Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; s=1600.; call hpfilter(gasout,gast,gasdev,s); call graph(gasout,gast,gasdev); call hpfilter(gasout,gast2,gasdev2,0.0); call tabulate(gasout,gast,gasdev,gast2,gasdev2); b34srun; Extended example showing effect of dropping obs. Also RATS runs as a check /; /; Illustrates testing against rats /; If obs are dropped off near end the ends of the series /; transformed series are changed. This suggests that /; models updated by more data points will ayve older /; observations changed. /; -------------------------------------------------------- %b34slet runrats=0; b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; s=1600.; call hpfilter(gasout,gast,gasdev,s); call graph(gasout,gast,gasdev :nolabel); * This gives the same answer as no filtering; call hpfilter(gasout,gast2,gasdev2,0.0); call tabulate(gasout,gast,gasdev,gast2,gasdev2); call print(' ':); call print('Dropping obs changes filter!!!':); call print('------------------------------':); newgas1= droplast(gasout,100); newgas2= droplast(gasout,10); call hpfilter(gasout, ngast, ngasdev,s); call hpfilter(newgas1,ngast1, ngasdev,s); call hpfilter(newgas2,ngast2, ngasdev,s); call tabulate(ngast1,ngast2, ngast); b34srun; %b34sif(&runrats.ne.0)%then; b34sexec options open('rats.dat') unit(28) disp=unknown$ b34srun$ b34sexec options open('rats.in') unit(29) disp=unknown$ b34srun$ b34sexec options clean(28)$ b34srun$ b34sexec options clean(29)$ b34srun$ b34sexec pgmcall$ rats passasts pcomments('* ', '* Data passed from B34S system to RATS', '* ', 'display @1 %dateandtime() @33 ', "display ' Rats Version ' %ratsversion()", '* ') $ PGMCARDS$ * filter(type=hp,tuning=1600) gasout / gasoutt set detrend = gasout-gasoutt print * * gasout gasoutt detrend b34sreturn$ b34srun $ b34sexec options close(28)$ b34srun$ b34sexec options close(29)$ b34srun$ b34sexec options /$ dodos(' rats386 rats.in rats.out ') dodos('start /w /r rats32s rats.in /run') dounix('rats rats.in rats.out')$ B34SRUN$ b34sexec options npageout WRITEOUT('Output from RATS',' ',' ') COPYFOUT('rats.out') dodos('ERASE rats.in','ERASE rats.out','ERASE rats.dat') dounix('rm rats.in','rm rats.out','rm rats.dat') $ B34SRUN$ %endif; HP_BP_1 Hodrich-Prescott and Baxter-King Analysis call hp_bp_1(julian,series,name,highfreq, lowfreq,nterms,lamda, print,graphit,rjulian,rseries,hptrend, hpdev,bptrend,bpdev); Performs Hodrick - Prescott and Baxter King Analysis julian = Julian date. If not available pass series of zero same length as series series = Input series name = Character object of name highfreq = Barter-King High Freq (6.) lowfreq = Baxter-King Low Freq (32.) nterms = # of terms for Baxter - King lamda = Hodrick-Prescott Lamda 1600. print = 0 => nothing, ne 0 => print graphit = 0 => nothing, ne 0 => graph rjulian = Revised julian rseries = Revised series hptrend = Hodrick-Prescott trend hpdev = Hodrick-Prescott dev bptrend = Baxter-King trend bpdev = Baxter-King dev HP_BP_1 is a subroutine from matrix2.mac. It must be loaded with call load(hp_bp_1); Test Case: HP_BP_1 HP_BP_2 Baxter-King & Hodrick-Prescott Moving Filtering call hp_bp_2(julian,series1,series2,nwindow,ncc, highfreq,lowfreq,nterms,lamda,njulian, cortrhp,cordevhp,cortrbp,cordevbp, var1trh,var2trh,var1devh,var2devh, var1trb,var2trb,var1devb,var2devb, corrmat1,corrmat2,corrmat3,corrmat4); Hodrick-Prescott and Baxter King Analysis on two series for a moving period. The estimated Hodrick - Prescott Series are truncated BEFORE variances and correlations are calculated. julian => Julian date. If not available pass series of zero same length as series series1 = Input series series2 = Input series nwindow = number in window ncc = # of lags for cross correlations highfreq = Barter-King High Freq (6) lowfreq = Baxter-King Low Freq (32) nterms = # of terms for Baxter - King lamda = Hodrick-Prescott Lamda njulian = Revised julian vector cortrhp = Correlation of trend HP data cordevhp = Correlation of dev HP data cortrbp = Correlation of trend BP data cordevbp = Correlation of dev BP data var1trh = Variance of series 1 trend HP data var2trh = Variance of series 2 trend HP data var1devh = Variance of series 1 dev HP data var2devh = Variance of series 2 dev HP data var1trb = Variance of series 1 trend BP data var2trb = Variance of series 2 trend BP data var1devb = Variance of series 1 dev BP data var2devb = Variance of series 2 dev BP data corrmat1 = Correlation matrix for trend HP data corrmat2 = Correlation matrix for dev HP data corrmat3 = Correlation matrix for trend BP data corrmat4 = Correlation matrix for dev BP data HP_BP_2 is a subroutine from matrix2.mac. It must be loaded with call load(hp_bp_2); Test case: HP_BP_2 HP_2 Hodrick - Prescott Moving Filtering call hp_2(series1,series2,nwindow,ncc,lamda,cortrhp, cordevhp,var1trh,var2trh,var1devh,var2devh, corrmat1,corrmat2,corrmat3,corrmat4); Performs Hodrick - Prescott Anlysis on two series for a moving period series1 = Input series series2 = Input series nwindow = number in window ncc = # cc lamda = Hodrick-Prescott Lamda cortrhp = Correlation of trend HP data cordevhp = Correlation of dev HP data var1trh = Variance of series 1 trend HP data var2trh = Variance of series 2 trend HP data var1devh = Variance of series 1 dev HP data var2devh = Variance of series 2 dev HP data corrmat1 = Correlation matrix for trend HP data corrmat2 = Correlation matrix for dev HP data HP_2 is a subroutine from matrix2.mac. It must be loaded with call load(hp_2); Example: b34sexec options ginclude('gas.b34'); b34srun; b34sexec matrix; call loaddata; call load(hp_2); call print(hp_2); julian=array(norows(gasin):); nwindow=50; ncc=10; lamda=100.; series1=gasin; series2=gasout; call echooff; call hp_2(series1,series2,nwindow,ncc, lamda,cortrhp,cordevhp,var1trh,var2trh, var1devh,var2devh,corrmat1,corrmat2, corrmat3,corrmat4); call names; call graph(var1trh,var1devh); b34srun; Test case: hp_2 IALEN Gen actual length of a buffer of character data call ialen(charvar,ilen) Gets actual length of a character*1 array or character*8 string charvar = Character*1 string ilen = position of last character Example: b34sexec matrix; call character(cc,'This ends at 15 '); call ialen(cc,ipos); call print('Should be 15',ipos); b34srun; IBFOPEN Open a file for Binary I/O call ibfopen('filename',accesscode,ihandle) Open a file for Binary I/O Examples: call ibfopen('test.ff',READONLY,ihande); call ibfopen('test.ff',WRITEONLY,ihandle); call ibfopen('test.ff',READWRITE,ihandle); Open file test.ff for readonly, writeonly and readwrite respectively. The access commands READWRITE and READ require existing files. The access command WRITEONLY will remove any data in the file prior to writting. Note: The file open must be accessed with the binary file I/O subroutines: call IBFOPEN('name',access,ihandle) call IBFCLOSE(ihandle); call IBFREADR(ihandle,rbuffer,ntoread,nread); call IBFREADC(ihandle,cbuffer,ntoread,nread); call IBFSEEK(ihandle,ipos,method); call IBFWRITER(ihandle,rbuffer,ntowrite,nwrite); call IBFWRITEC(ihandle,cbuffer,ntowrite,nwrite); Properly used, these commands allow reading and processing of a large number of file types. Extensive example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFCLOSE Close a binary file that was opened by IBFOPEN call ibfclose(ihandle); Closes a file that was currently open. Extensive example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFREADR Read a Real*1 value from a binary file call ibfreadr(ihandle,rbuffer,ntoread,nread) Read a real*1 value from a binary file ihandle => File handle from ibfopen rbuffer => Real buffer ntoread => Number of bytes to read nread => Actual number read ibfseek can be used to position the read/write pointer Extensive example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFREADC Read a Character*1 value from a binary file call ibfreadc(ihandle,cbuffer,ntoread,nread) Reads a character*1 value from a binary file ihandle => File handle from ibfopen rbuffer => Real buffer ntoread => Number of bytes to read nread => Actual number read ibfseek can be used to position the read/write pointer Extensive example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFSEEK Position Binary read/write pointer call ibfseek(ihandle,ipos,method) Positions the read/write pointer ihandle => File handle from ibfopen ipos => required position to read/write. on exit ipos set to updated position method => set as: FROMSTART FROMCURRENT FROMEND Use of IBFSEEK allows random access of a binary file. Extensive example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFWRITER Write noncharacter buffer on a binary file call ibfwriter(ihandle,rbuffer,ntowrite,nwrite) Write a noncharacter buffer of a binary file ihandle => File handle from ibfopen rbuffer => Real buffer ntoread => Number of bytes to write nread => Actual number of bytes written ibfseek can be used to position the read/write pointer Example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IBFWRITEC Write character buffer on a binary file call ibfwritec(ihandle,cbuffer,ntowrite,nwrite) Write character buffer on a binary file ihandle => File handle from ibfopen cbuffer => character buffer ntoread => Number of bytes to write nread => Actual number of bytes written ibfseek can be used to position the read/write pointer Example b34sexec matrix; /$ /$ Tests both Character and real reading and writting /$ call ibfopen('test.ff',writeonly,ihandle); x=rn(array(10:)); j=norows(x)*8; call ibfwriter(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.ff',isize); call print('The file size for test.ff is ',isize); xnew=array((isize/8)+1:); call ibfopen('test.ff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call ibfreadr(ihandle,xnew,isize,ii); call tabulate(x,xnew); call ibfclose(ihandle); /$ /$ Character Tests /$ call ibfopen('test.cff',writeonly,ihandle); call character(x,'abcdefghi'); j=norows(x); call ibfwritec(ihandle,x,j,iwrite); call print('Number of bites written ',iwrite); call names(all); call ibfclose(ihandle); call ifilesize(' ','test.cff',isize); call print('The file size for test.cff is ',isize); xnew=rtoch(array((isize/8)+1:)); call character(cnew,xnew); call ibfopen('test.cff',readonly,ihandle); ipos=0; call ibfseek(ihandle,ipos,fromstart); call names(all); call print(cnew); call ibfreadc(ihandle,cnew,isize,ii); call print(x,cnew); call ibfclose(ihandle); call dodos('erase test.ff'); call dounix('rm test.ff'); b34srun; IB34S11 Parse a token using B34S11 parser call ib34s11(string,ibase,ifbase,isize,itokty,inewp,imax) Parses a character*1 array string string => Character*1 string to parse from ibase to imax. If IMAX = 0 uses end of string. ibase => Where to start looking in string ifbase => Location of token. = 0 if not found isize => Size of token. If isize=0 => no token found. itokty => Token type: 0 = unknown 1 = $ 2 = ( 3 = ) 4 = = 5 = integer value 6 = real value 7 = var name (coded 77 if between " " and ' ') 7 is also a var string 8 = opt (such as x=log(z)$ ) 88 log{8.} 888 log[8.] 9 = parm (parm=key or parm=(key1,key2) ) 10 = * 11 = - 12 = + 13 = b34send 14 = / 15 = : 16 = ; 17 = . 18 = ' 19 = " 20 = logical operator 21 = | 22 = { 23 = } 24 = [ 25 = ] 26 = @ 27 = , inewp => New pointer if the string has space left. =-99 if done. imax => Upper limit to look at. If set = 0 then the max of string used. Note: #, & and _ are not special characters. Note: The b34s11 routine is the main b34s parser/tokenizer. Use of this routine allows the expert programmer to parse a line and detect what is there, and decide on the next step quickly. This command is of use for a developer. Look at the getr16 and getr8 routines in staging2.mac for how this routine might be used. Example: b34sexec matrix; call character(cc,'10. 11 test y(10) jj=44 print'); ibase=1; call echooff; do j=1,100; imax=0; call ib34s11(cc,ibase,ifbase,isize,itokty,inewp,imax); if(isize.eq.0)go to finish; call print('ifbase found ',ifbase :line); call print('Size of token found ',isize :line); call print('Type of token found ',itokty :line); call print('inewp of token found ',inewp :line); i=integers(ifbase,ifbase+isize-1); find=cc(i); call character(tt,'Token found was: '); call expand(tt,find,20,(20+isize)); call print(tt :line); call print(' ' :line); ibase=inewp; if(inewp.eq.-99)go to finish; enddo; finish continue; call print('All done tokenizing'); b34srun; Application loading data using getr16 routine. /$ /$ Reads a character array into real*16 and real*8. /$ Tests input /$ b34sexec matrix; call character(cc,' 1 0 6 1 63 2 364 3 1365 4 3906 5 9331 6 19608 7 37449 8 66430 9 111111 10 177156 11 271453 12 402234 13 579195 14 813616 15 1118481 16 1508598 17 2000719 18 2613660 19 3368421 20'); call load(ntokin :staging); call load(getr16 :staging); call echooff; call ntokin(cc,nfind,0,ibad); call getr16(cc,nfind,x16,ibad); /$ repack xm=matrix(nfind/2,2:x16); call print(xm); b34srun; IFILESIZE Determine number of bites in a file call ifilesize('dir','fname',isize) Determine number of bites in a file 'dir' => Directory of file 'fname' => File name isize => size if bytes of file. isize=0 if file not found Example b34sexec matrix; call ifilesize('c:\b34slm','gas.b34',isize); call print(isize); b34srun; IFILLSTR Fill a string with a character call ifillstr(string,chr) Fill a string with a character string => String to fill char => Character to place in string Example: b34sexec matrix; call character(cc,'This is a string'); newcc=cc; call ifillstr(newcc,'a'); call print(cc,newcc); b34srun; IGETICHAR Obtain ichar info on a character buffer call igetichar(charvar,ival) Obtain ichar info on a character buffer charvar => Character*1 variable ival => Integer*4 array of size iend-istart+1 Example b34sexec matrix; call character(astring,'ABCDEFG'); call igetichar(astring,ichar); ichar2=ichar+1 call igetchari(ichar2,newstr); call print(astring,ichar,ichar2,newstr); b34srun; IGETCHARI Get character from ichar value call igetchari(ival,charvar) Get character from ichar value ival => Integer vector of ichar values charvar => Characters from ival Example b34sexec matrix; call character(astring,'ABCDEFG'); call igetichar(astring,ichar); ichar2=ichar+1 call igetchari(ichar2,newstr); call print(astring,ichar,ichar2,newstr); b34srun; IJUSTSTR Left/Right/center a string call ijuststr(string,task) Left/Right/center a string string => String to operate on task => task is LEFT, CENTER, RIGHT Example: b34sexec matrix; call character(c,'This is a statement '); leftc=c; centerc=c; rightc=c; call ijuststr(leftc, left); call ijuststr(centerc,center); call ijuststr(rightc, right); call print(c,leftc,centerc,rightc); b34srun; ILCOPY Move bites from one location to another call ilcopy(nbytes,in,inc1,instart,out,inc2,ioutstart) Move bites from one location to another nbytes => # of bytes to move in => input variable inc1 => incrument for in instart => byte to start with for in out => out variable inc2 => incrument for out variable ioutstart => byte to start with for out Variables IN and OUT must be real*8, integer*4 or real*4. Warning: Do not use subscripted variable for out variable since it will replaced by a temp and NOT copied. This command allows exact placement of bits within an array and is able to by pass the usual Fortran copy. The ILCOPY command is intended for the expert programmer. Example: b34sexec matrix; * Put in reals we know what they are; x=array(20:integers(20)); call print(x); call displayb(x); x(1)=0.0; x(2)=1.0; * Hide an integer in a real; call displayb(x); i1=1; i2=2; call ilcopy(4,i1,1,1,x,1,1); call ilcopy(4,i2,1,1,x,1,5); call displayb(x); b34srun; Note: If Character*1 data is need to be moved, use EXPAND and CONTRACT. In place replacement can be done with: b34sexec matrix; * we want aabb at 5-8 in cc; * We do not want to expand; call character(cc,'This is a test'); call character(new,'aabb'); call contract(cc,5,8); call expand(cc,new,5,8); call print(cc); b34srun; ILOCATESTR Locate a first non blank character call ilocatestr(string,ipos) Cocate first non blank character string => Character*1 string to search ipos => position of string Example: b34sexec matrix; call character(cc,' in5to11 '); call ilocatestr(cc,in,iout); call print(cc,in,iout); b34srun; ILOWER Lower case a string - 500 length max call ilower(string) Lower case a string - 500 length max string => Character*1 array to lower case. Max length 500. Example: b34sexec matrix; call character(cc,'THIS IS UPPER'); lower=cc; call ilower(lower); upper=lower; call iupper(upper); call print(cc,lower,upper); b34sreturn; INEXTR8 Convert next value in string to real*8 variable call inextr8(string,real8val) Convert next value in string to real*8 variable string => Character*1 array of max length 500. real8val => Next real*8 value. If blank set to missing. Note: String is cleared. Example: b34sexec matrix; call character(cc,'2.3 5. 99 Bob'); call print(cc); call inextr8(cc,r8); call print(cc); call inextr4(cc,r4); call print(cc); call inexti4(cc,i4); call print(cc); call inextstr(cc,ss,ihave); call print(cc); call inextstr(cc,ss2,ihave2); call print(r8,r4,i4,ss,ihave,ihave2); b34srun; INEXTR4 Convert next value in string to real*4 variable call inextr4(string,real4val) Convert next value in string to real*4 variable string => Character*1 array of max length 500. real4val => Next real*4 value. If blank set -999999999. Note: String is cleared. Example: b34sexec matrix; call character(cc,'2.3 5. 99 Bob'); call print(cc); call inextr8(cc,r8); call print(cc); call inextr4(cc,r4); call print(cc); call inexti4(cc,i4); call print(cc); call inextstr(cc,ss,ihave); call print(cc); call inextstr(cc,ss2,ihave2); call print(r8,r4,i4,ss,ihave,ihave2); b34srun; INEXTSTR - Extract next blank deliminated sub-string from string call inextstr(string,substr,ihave) Extract next blank deliminated sub-string from a string string - Character*1 array of max length 500. substr - Character*1 array of substring. ihave - =0 if have a substring, =1 if do not. Example: b34sexec matrix; call character(cc,'2.3 5. 99 Bob'); call print(cc); call inextr8(cc,r8); call print(cc); call inextr4(cc,r4); call print(cc); call inexti4(cc,i4); call print(cc); call inextstr(cc,ss,ihave); call print(cc); call inextstr(cc,ss2,ihave2); call print(r8,r4,i4,ss,ihave,ihave2); b34srun; INEXTI4 - Convert next value in a string to integer call inexti4(string,intval) Convert next value in a string to integer string - Character*1 array of max length 500. intval - Next integer*4 value. If blank set -999999999 Note: String is cleared. Example: b34sexec matrix; call character(cc,'2.3 5. 99 Bob'); call print(cc); call inextr8(cc,r8); call print(cc); call inextr4(cc,r4); call print(cc); call inexti4(cc,i4); call print(cc); call inextstr(cc,ss,ihave); call print(cc); call inextstr(cc,ss2,ihave2); call print(r8,r4,i4,ss,ihave,ihave2); b34srun; INTTOSTR - Convert integer to string using format call inttostr(int,str,fmt) Convert integer to string using format int - integer str - string fmt - up to 8 characters of format Example: b34sexec matrix; call inttostr(88,is88,'(i4)'); call character(cc,'99.88D32'); call istrtor8(cc,bigr8); call character(cc,'77'); call istrtoint(cc,is77); xx=99.99; call ir8tostr(xx,is99p99,'(g12.4)'); call print(is88,bigr8,is77,is99p99); b34srun; IR8TOSTR - Convert real*8 value to string using format call ir8tostr(real8,str,fmt) Convert real*8 value to string using format real8 - Real*8 value str - string fmt - up to 8 characters of format Example: b34sexec matrix; call inttostr(88,is88,'(i4)'); call character(cc,'99.88D32'); call istrtor8(cc,bigr8); call character(cc,'77'); call istrtoint(cc,is77); xx=99.99; call ir8tostr(xx,is99p99,'(g12.4)'); call print(is88,bigr8,is77,is99p99); b34srun; IRF - Impulse Response Functions of VAR Model call irf(x,ibegin,iend,beta,t1,res,sigma,psi,ipsi,tirf, iprint1,nterms,nlag,var,varxhat,rsq); Calculates Transfer Function Impulse Response Function (TFIRF) of a VAR Model and the VMA form of the IRF model. Assume A(L)X=e Transformed model is: X = PSI(L)e where PSI(L) = INV(A(L)) ------------------------------------------------------- This is in contrast to the Transfer function form of the model tirf that is calculated by normalizing by diagonal polynomial. Assume VAR Model for 2 Series X1 & X2: Orig. eq: 1 b11(L)X1 + b12(L)X2 = e1 Orig. eq: 2 b21(L)X1 + b22(L)X2 = e2 Trans. eq: 1 X1 + [b12(L)/b11(L)]X2 = [1/b11(L)]e1 Trans. eq: 2 X2 + [b21(L)/b22(L)]X1 = [1/b22(L)]e2 If 1 to k-1 cols are multiplied by -1 we get the alternative tirf form Alt. eq: 1 X1 = -[b12(L)/b11(L)]X2 + [1/b11(L)]e1 Alt. eq: 2 X2 = -[b21(L)/b22(L)]X1 + [1/b22(L)]e2 For one variable case [1/b11(L)] is psi(L) --------------------------------------------------------------- Arguments: x - n by k matrix of the series in var model ibegin - Start of period iend - End of period beta - Coef saved as a matrix(k,k*nlag+1) use polymdisp to display if remove constant t1 - t stat saved as a matrix(k,k*nlag+1) use polymdisp to display if remove constant sigma - Variance Covariance of errors period psi - psi weights saved in byorder form ipsi - index to read psi weights tirf - Transfer function form of IRF saved in byorder form which is the same as psi 1,1 position shows effect of variable 2 on variable 1 1,2 position effect of noise iprint1 - set 1 => print estimation results Coef by order