16 November 2018

Anatomy of a procedure (1)

Only recently the IDE features ‘Proc Disassembly’, an option available under the Edit | Proc menuitem. This is a valuable resource if you want to get a better understanding of the code generated by the compiler. Once you understand the disassembly of a proc you can use the information to your advantage, especially when it comes to optimizing procedures.

Bare minimum: Naked
Let’s start with a Naked procedure. A Naked procedure is fully optimized, both in size as in performance. This comes with a penalty though, a Naked procedure lacks support for dynamic variables, structured exception handling, and runtime debugging (Tron, Trace). The Naked attribute forces the compiler to produce code much like if it would be done in a pure assembly program. The assembly code of a procedure has great similarity with textbook samples. It’s not hard to understand the procedure flow when it is compared to the theory in the assembly books. Therefor, I start this series on the anatomy of procedures with these bare minimum procs. Examining naked procedures allow us to understand how a proc is constructed and this knowledge can later be used to examine regular procedures.

The following sample shows a Naked proc taking two parameter of a simple type (Long). For now, we’ll omit the use of dynamic datatypes like String, Variant, Object, etc. The procedure contains the local variable tmp, also of a simple datatype, and assigns the product of x and y to tmp. This is the entire program:

TestMul(2, 3)
Proc TestMul(x As Int, y As Int) Naked
  Local Int tmp
  tmp = x * y

Now put the caret inside the procedure TestMul and select Proc | Disassembly, it produces the following listing in the debug output window:

--------  Disassembly -----------------------------------
1 Proc TestMul(x As Int, y As Int) Naked (Lines=5)
042704D0: 53                             push    ebx
042704D1: 55                             push    ebp
042704D2: 8B EC                          mov     ebp,esp
042704D4: 8D 5D 80                       lea     ebx,-128[ebp]
042704D7: 2B C0                          sub     eax,eax
042704D9: 50                             push    eax
042704DA: DB 45 0C                       fild    dpt 12[ebp]
042704DD: DA 4D 10                       fimul   dpt 16[ebp]
042704E0: DB 5B 7C                       fistp   dpt 124[ebx]
042704E3: 8B E5                          mov     esp,ebp
042704E5: 5D                             pop     ebp
042704E6: 5B                             pop     ebx
042704E7: C2 08 00                       ret     8

The first line specifies the line number of the procedure (1), its entire prototype, and the number of lines (here 5, but might be more if the procedure includes any trailing empty lines).
The numbers at the start of each line show the memory address of the instructions, which might be different from your result. Consequently, in this case, the function ProcAddr(TestMul) would return the address of the first byte of the procedure: 0x042704D0.
After the memory address follow the opcodes for the assembly instruction. For instance, the opcode with value 0x53 corresponds to the push ebx assembly command. Some instructions  require a one byte opcode only, others require multiple opcodes.

The first 6 lines make up the the procedure’s entry code (sometimes called prologue). The last 4 lines are the procedure’s exit code (or epilogue). The lines in between represent the actual functionality of the procedure.

Entry code
The procedure’s entry code prepares the procedure’s code to handle parameters and local variables:

push    ebx            ; save ebx
push    ebp            ; save ebp
mov     ebp,esp        ; establish stackframe
lea     ebx,-128[ebp]  ; let ebx reference local vars
sub     eax,eax        ; eax = 0
push    eax            ; clear first local var

Whenever a procedure takes a parameter or declares a local variable you’ll always find the same three instructions at the start of each procedure: push ebx / push ebp / mov ebp, esp. If the procedure also contains local variables the fourth line lea ebx, –128[ebp] is present as well. Following this line you’ll find the code that initializes the local variable; all local variables are initialized to zero.

Local variables, the purpose of ebx
In GFA-BASIC 32 the ebx register has a special purpose and thus ebx cannot be used as a general purpose register. It is used as a fixed reference point to address the local variables.

Note - According to the documentation it allows to layout variables that require more than 4 bytes (Double, Date, Large, Currency) on 8-byte borders increasing performance when accessed.

The ebx value points to an address 128 bytes down on the stack relative to the value in ebp, the stackframe. Although the first local variable is actually located at ebp – 4 , it will be referenced using the value in ebx. The location of the local variable is +124 bytes relative to the value in ebx, in assembly syntax the tmp variable is located at 124[ebx].

The value stored at that position is obtained using dword ptr 124[ebx]. This is illustrated by the next three lines of code where the parameters are multiplied by the fpu (floating-point processor) and where the result is assigned to the local variable tmp.

042704DA: fild    dpt 12[ebp]  ; load value of param x into fpu
042704DD: fimul   dpt 16[ebp]  ; multiply by value in param y
042704E0: fistp   dpt 124[ebx] ; store result in tmp

The parameters x and y are accessed using the value in ebp as we will see.

Stack structure
When the procedure is called, the caller puts the parameters y and x on the stack, in reversed order. GB subroutines conform to the stdcall convention, which means that the parameters are pushed from right to left and that the subroutine corrects the stack before returning. Since y is the most right parameter it is pushed first, followed by the parameter at the left (here x). Then the CPU adds the return address on the stack and executes the subroutine.

From this point on the stack is prepared according the procedure’s entry code discussed above. The result can be viewed in the next picture:

The entry code saves the current values from the ebx and ebp registers on the stack. Then ebp is assigned the new esp value. Now ebp is used to address the parameters: parameter x is located at a positive offset of 12 bytes from the value in ebp; in assembly code 12[ebp]. Parameter y is 16 bytes up the stack relative to the value in ebp, in assembly code 16[ebp].

To address the parameters throughout the procedure ebp needs to remain constant during the execution of the procedure. The same is true for ebx that is used to address the local variables. We cannot use esp to reference both parameters and local variables because esp changes automatically during the execution of the procedure. (Although C/C++ compilers sometimes keep track of esp and address all stack variables using an offset to esp.)

Allocating and initializing
After the stackframe is established (mov ebp, esp) the next step requires the reservation of stackspace for local variables, see listing. The general idea, and described in most textbooks, is to subtract the required number of bytes from esp and then clear that piece of memory. In our sample esp would have to be decreased by 4 bytes (for the Long variable tmp) and then cleared by zero. Although GB produces the same effect, it proceeds a bit different.

Note that the first byte of the 32-bits local variable tmp is located at ebp-4. After creating the stackframe by mov ebp, esp the registers esp and ebp point to the same stack address. To reserve and initialize the 4 bytes below esp GB uses the instructions sub eax, eax / push eax.

Subtracting a register by itself results in zero. By pushing zero, now the value in eax, GB both reserves and initializes the local variable in one step. It prevents the additional step to first decrease esp explicitly. The technique to use push to reserve and initialize is typical for GB. The push eax can be repeated to clear and reserve all stack memory necessary for local variables. Thus, if the procedure would have contained two local variables of type long it would have had two push eax instructions.

Exit code
Before leaving the procedure the stack must be returned to the state it was when the procedure was entered. In addition, because of the stdcall convention, the procedure must remove the bytes necessary for the parameters (2 * 4 bytes for two long parameters). This is how its done:

042704E3:   mov     esp,ebp ; restore esp
042704E5:   pop     ebp     ; restore ebp
042704E6:   pop     ebx     ; restore ebx
042704E7:   ret     8       ; return, discarding parameters

When the program returned to the caller the registers that matter and must remain constant are restored. This makes sure the caller can use the correct ebp value to access its parameters and that ebx can be used to access its local variables.

Optimize using disassembly
Inspecting a procedure’s disassembly is useful to get an idea what’s going on underneath the GFA-BASIC statements. The example presented in this blog proves why. The example performs a multiplication of two integer parameters and stores the result in another integer. As you can see, the compiler generates floating-point assembly instructions to perform the math. Since all variables are of type Long, the compiler could have generated more efficient code using the integer multiplication instruction imul. However, the compiler generates integer instruction only for addition and subtraction operators. Now its up to the programmer to optimize this procedure by replacing the multiplication operator * by the Mul operator. The optimized procedure then becomes:

Proc TestMul(x As Int, y As Int) Naked
  Local Int tmp
  tmp = x Mul y

Now compile the code and inspect the disassembly. As you can see the floating point instructions are vanished and replaced by the imul instruction.

Inspecting a procedure’s disassembly requires knowledge to identify the three parts  of a procedure; the entry code, the actual code, and the exit code, We discussed how to identify parameters and local variables and saw how GB uses a specific technique to reserve and initialize local variables.

In coming blog posts we’ll discuss non-naked procedures and how you can tell a procedure is a good candidate to be naked.

09 September 2018

Did the mouse leave the window?

There are two mouse-messages that are never received unless you explicitly instruct Windows to track the mouse movement. The first message is WM_MOUSELEAVE that is supposed to report that the mouse has left the client-area. The second is WM_MOUSEHOVER which is posted after hovering a certain amount of time over some area. To obtain one (or both) of these messages you need to call TrackMouseEvent() API which notifies the application when the mouse leaves the window or when the mouse hovers over an area for a while.

The next program illustrates how to implement a mouseover feature by drawing a box that turns black when you move the mouse over it. The basic idea is to use WM_ MOUSEMOVE to know when the mouse has moved in or out of the box. The only problem is that if the user moves the mouse quickly outside the window, you won't get a WM_ MOUSEMOVE. To implement a correct behavior of mouseover, you need to know when the mouse has left the window entirely.

The program doesn’t use the _MouseMove eventsub, but combines the mouseover-logic into the _Message eventsub, which receives all (posted) mouse messages. There are no event subs for WM_MOUSELEAVE and WM_MOUSEHOVER, so they have to be handled in a general event- sub. An alternative would be to handle the messages in _MessageProc, but its use is a bit more complicated. In addition, _Message doesn’t require any return values, so it serves our purpose best.

OpenW Center 1
Until Win_1 Is Nothing

Sub Win_1_Paint
  Box 10, 10, 100, 100

Sub Win_1_Message(hWnd%, Mess%, wParam%, lParam%)
  Static Bool fTrackingMouse, fBoxHighLighted
  Local Int mx, my
  Switch Mess%

    ' Track a mouseleave event. Results in a WM_MOUSELEAVE
    ' message when the mouse leaves the window.
    If !fTrackingMouse           ' set it only once
      tme.cbSize     = SizeOf(TRACKMOUSEEVENT)
      tme.dwFlags    = TME_LEAVE
      tme.hwndTrack  = Me.hWnd
      fTrackingMouse = TrackMouseEvent(tme) != 0

    ' If mouse is over the box start timer
    mx = LoWord(lParam%), my = HiWord(lParam%)
    If mx > 10 && mx < 100 && my > 10 && my < 100
      tme.cbSize      = SizeOf(TRACKMOUSEEVENT)
      tme.dwFlags     = TME_HOVER       ' start timer
      tme.hwndTrack   = Me.hWnd
      tme.dwHoverTime = HOVER_DEFAULT   ' use default time
      TrackMouseEvent(tme)              ' now wait for WM_MOUSEHOVER
    Else If fBoxHighLighted             ' hilighted and not over box
      Win_1.Invalidate 10, 10, 90, 90
      fBoxHighLighted = False

  Case WM_MOUSELEAVE            ' triggered by TrackMouseEvent
    fTrackingMouse = False      ' TrackMouseEvent not active anymore
    If fBoxHighLighted          ' redraw original box
      Win_1.Invalidate 10, 10, 90, 90
      fBoxHighLighted = False   ' box is not highlighted

  Case WM_MOUSEHOVER            ' triggered by TrackMouseEvent's timer
    mx = LoWord(lParam%), my = HiWord(lParam%)    ' mouse coordinates
    If !fBoxHighLighted && mx > 10 && mx < 100 && my > 10 && my < 100
      PBox 10, 10, 100, 100                     ' highlight the box
      fBoxHighLighted = True

  - DWord  cbSize
  - DWord  dwFlags
  - Handle hwndTrack
  - DWord  dwHoverTime
Declare Function TrackMouseEvent Lib "user32" Alias _
  "TrackMouseEvent" (ByRef EventTrack As TRACKMOUSEEVENT) As Long

The _Message sub declares two static booleans, fTrackingMouse and fBoxHighLighted, that keep track of the current state of the mouseover-logic. (If I would have used the _MouseMove eventsub to initiate the mouse tracking the variables should have been declared global, I always try to avoid global variables as much as possible.)
When the first (of many) WM_MOUSEMOVE message is received, the TrackMouseEvent() API is used to set up a WM_MOUSELEAVE  "one-shot" event. Exactly one and only one  WM_MOUSELEAVE message will be posted to the window specified in the hwndTrack member of the TRACKMOUSEEVENT structure, when the mouse has left the client area.
Note - The message will be generated only once. The application must call the TrackMouseEvent API again in order for the system to generate another WM_MOUSELEAVE message. In addition, when the mouse pointer is not over the application, a call to TrackMouseEvent() will result in the immediate posting of a WM_MOUSELEAVE message.

When the mouse is over the box, the TrackMouseEvent() is used to start a timer that eventually posts the WM_MOUSEHOVER message. After receiving WM_MOUSEHOVER the box is highlighted if the mouse is still over the box. The fBoxHighLigted variable is set to indicate the state of the box. If the variable is set but the mouse is no longer over the box the area occupied by the box is invalidated so that it is redrawn eventually.

21 August 2018

What’s the difference between ?: and Iif()?

There is a simple answer: none. The conditional ?: is called a ternary operator because it takes three arguments. It is often used as a shortcut for an If – Else statement. The syntax is:

result = condition ? expr1 : expr2

The condition must evaluate to either True or False. If the condition evaluates to True, expr1 is assigned to result. If the condition is False, result becomes the value of expr2. Consequently the data-types of expr1 and expr2 must match the type of result, they are either equal or the types of expr1 and expr1 can be implicitly converted to the type of result. Here is an example of how to use the ternary operator:

Dim number As Int, result As String
number = 1
result = number >= 0 ? "Positive" : "Negative"

This construct replaces the following If-Else statement:

If number >= 0
  result = "Positive"
  result = "Negative"

The ?: operator is ‘inlined’, the compiler generates the fastest code possible without calling any function. Nonmatching data types are catched by the compiler.
The If-Else statement generates a little more code, because it contains two assignment statements.

The Iif() function is implemented in exactly the same way as the ?: operator. It generates the exact same inlined code.

result = Iif(number >= 0, "Positive", "Negative")

This results in the same optimized code:

037C04D0: mov     eax,[0x048BC190]  ; number to eax
037C04D5: test    eax,eax
037C04D7: jl      short 0x037C04E0  ; jmp if < 0
037C04D9: mov     eax,0x049205B4    ; address of “Positive”
037C04DE: jmp     short 0x037C04E5  ; goto
037C04E0: mov     eax,0x049205F4    ; address of “Negative”
037C04E5: push    eax               ; address of string
037C04E6: push    0x02A37F90        ; address of result
037C04EB: scall   STOSTRSV          ; store string in result

The disassembly is produced with the new Disassemble Proc feature present in the recent updates of GFA-BASIC.
Note – the compiler setting Branch-Optimizing is set to ‘Normal’, which results in shorter code.

29 April 2018

Definition of KB changed

Being an old-school programmer I learned 1KB == 1024 bytes, however this has been changed back in 1998. The prefix K (kilo) now means 1000 not 1024: 1 kilo byte is 1000 bytes. The why is discussed at physics.nist.gov and of course wikipedia.

The latest GB32 update displays the size of the created EXE (Gll or Lg32) after it has been written to disk. The number of KB is calculated as filesize / 1024. However, the value should have added the KiB unit, rather than the KB unit. The KiB unit denotes a value divided by 1024, KB doesn’t. If the file size is to be displayed in the KB unit it should have been divided by 1000.

I became aware of this when I noticed the Explorer displayed a different size for the compiled file. The next update will fix this by displaying the correct value in KB, so filesize /1000.

30 March 2018

Floating-point numbers

Often floating-point numbers lead to confusion and frustration. Unfortunately, these problems cannot be avoided and to properly work with floating-point values a basic understanding is required.

How floating-point numbers are stored
Floating point decimal values generally do not have an exact binary representation. This is a side effect of how the FPU represents and processes floating point numbers. The storage format for Double and Single is the same as expected by the FPU-registers of the CPU. This ensures consistency and fast reading from and writing to memory. The problem however, is how to store a floating-point value in a binary computer. This is solved by storing a floating-point number as a formula. There are two types of floating-point numbers: Float (or Single) and Double. The difference is their size in bytes, and therefore the minimum and maximum values that can be stored. Another difference is the higher accuracy for a Double. The maximum number a Float (taking 4-bytes) can store is much less than a Double (taking 8-bytes) can store. Since floating-point numbers can have an infinite number of values, you cannot store all of them in either 4 bytes (float) or 8 bytes (double). To be able to store as much numbers as possible, with as much accuracy as possible, another approach is necessary. A floating-point value is stored as a formula:

X = (-1)^sign * 2^(exponent - bias) * (1 + fraction * 2^-23)

The formula contains 3 variables (sign, exponent, fraction) and one constant (bias).The bias for single-precision numbers is 127 and 1,023 (decimal) for double-precision numbers. The values of these formula-variables are stored in either 4 bytes for a Single or 8 bytes for a Double.

The next example uses an user defined type Sfloat to illustrate the storage of the Float data type. The values for fraction and exponent, together with a sign bit, are stored in the 4 bytes. By using a Union we can assign a value to a Single variable and then use Sfloat to dump the 4 bytes that make up the Float:

Type Sfloat
  fraction As Bits 23   // fractional part
  exponent As Bits  8   // exponent + 127
  sign     As Bits  1   // sign bit
Type TFloat Union       // sizeof() = 4
  value As Float
  sf    As Sfloat
Dim fv As TFloat
fv.value = 2.0     : DumpFloat(fv)
fv.value = 0.0     : DumpFloat(fv)
fv.value = -345.01 : DumpFloat(fv)
' Do test some more ..

Proc DumpFloat(ByRef tflt As TFloat)
  Global Const bias As Int = 127        ' Standard IEEE
  Global Const frexp As Float = 2 ^ -23 ' Standard IEEE
  Dim flt!

  Debug "> DumpFloat:"; tflt.value;
  With tflt.sf
    Debug " (sign =";.sign; " exponent ="; .exponent; _
      " fraction =";.fraction;")"
    Debug "Binary format: ";Bin(.sign, 1)` _
      Bin(.exponent, 8)`Bin(.fraction, 23)

    ' Reconstruct value from Sfloat using formula:
    flt! = ((-1) ^ .sign Mul 2 ^ (.exponent - bias)) _
      * (1 + (.fraction * frexp))
    Debug "Float reconstructed ="; flt!

The output of the demo is:

> DumpFloat: 2 (sign = 0 exponent = 128 fraction = 0)
Binary format: 0 10000000 00000000000000000000000
Float reconstructed = 2
> DumpFloat: 0 (sign = 0 exponent = 0 fraction = 0)
Binary format: 0 00000000 00000000000000000000000
Float reconstructed = 0
> DumpFloat:-345.01 (sign = 1 exponent = 135 fraction = 2916680)
Binary format: 1 10000111 01011001000000101001000
Float reconstructed =-345.01

What does this tell us? A Float (or Double) is stored and described using 3 components in the bits of either a 4 or 8 bytes type. Due to the limited storage of all these components only an approximation of the decimal value can be ‘described’. When a floating point value is assigned to a variable the value is dissected into these 3 components. To get back to the original floating-point number these 3 components are substituted in this standardized formula.

Effect of floating point values
A floating-point value is stored by a description, not by its value. This makes it inherently inaccurate. Even common decimal fractions, such as 0.0001 cannot be represented exactly in binary, only fractional numbers of the form n/f where f is an integer power of 2 can be expressed exactly with a finite number of bits. Examples are 1/4, 7/16, 3/128, of each f is a power of 2.
The inaccuracy may increase slightly when a floating point value is loaded into a 80-bits FPU register. The FPU fills the remaining bits, because the 80-bits representation is different from the 32-bits Single or 64-bits Double format. Moving a value out of the FPU register will round the value back to fit in either a Single or a Double. Storing and transporting may add to the inaccuracy of of the value.

The following example shows what happens when the small error in representing 0.0001 propagates to the sum:

Dim dSum As Double, i As Int
For i = 1 To 10000
  dSum = dSum + 0.0001
Next i
Debug dSum     ' = 0.999999999999906

Theoretically the sum should be 1.0.

Not only the calculations suffer from inaccuracy, comparisons with floating point numbers are equally problematic. The following example demonstrates a ‘forbidden’ comparison between a floating-point constant number and the result of a calculation:

Global Double dVal1, dVal2
dVal1 = 69.82
dVal2 = 69.20 + 0.62
Assert dVal1 == dVal2  ' Not equal

This throws an ASSERT exception, because the assertion that dVal1 and dVal2 are equal fails.
A comparison between two floating-point constants of the same type is allowed. For instance, the GFA-BASIC runtime returns a Single constant from DllVersion (2.33; 2.341; etc.). This constant may be compared to a literal Single constant (note the exclamation mark, without it 2.33 is a double!):

If DllVersion == 2.33! MsgBox "This is version 2.33"

Never compare two different data types, a Single to to Double, or a floating-point to an integer, These comparisons will most certainly fail (unless they can be described using a finite number of bits, see above). Any comparison to a floating point will most likely fail, because the comparison is executed in the FPU expanding the values to 80-bits. The same is true for the comparison of the results of two floating point calculations, it will most certainly fail.

The most logical solution for floating-point comparison of type Double is the use of the special operator NEAR, which uses only 7 decimal digits from both expressions for the comparison. In practice the expressions are compared as if they are both of Single precision.

Improve floating-point consistency in calculations
If your application expects multiple fp-calculations, it is necessary to keep the intermediate values in the proper data format, otherwise small errors are propagated through the calculations. For multiple floating-point calculations the FPU uses the intermediate results that it holds in the 80-bits FPU registers. However, these 80-bits calculations do not reflect the data types involved, the Single or Double. Due to the extra level of accuracy multiple calculations may produce unexpected results. The compiler setting ‘Improve floating-point consistency’ inserts code to load and write immediate results from and to memory in the appropriate type. This decreases program speed, but improves the chance for an expected result of the calculation. Make sure the ‘Improve floating-point consistency’ is checked always (unless you know exactly what you’re doing).

Floating-point values are inherently inaccurate, you might want to avoid them as much as possible. Instead use integers when ever possible, or otherwise use Currency, which is an integer value as well. The Currency data type exactly stores up to 19 digits, with 4 digits after the decimal point.

21 March 2018

Function and Sub parameters

In the new English Html help additional information is provided for Function and Sub. Since this is new information a copy of the text has a place in a blogpost.

Function parameters
The return value of a Function can be assigned to a local variable with the same name as the Function. When the return type is a numeric data type a local variable of that type is automatically added to the function’s local variables. With String, Type (UDT) and Variant as the return type a by reference variable is passed as the last argument on the stack. The string, UDT or variant becomes the variable that can be used to pass the function’s return value. Therefor, the following is equal:

Dim h$
h$ = testf(8) ' assign result to h$
testp(8, h$)  ' put result in h$
Function testf(a%) As String testf = "3" & a%
Procedure testp(a%, ByRef p$) p$ = "3" & a%

In the function testf the variable h$ is silently passed on the stack. Inside the function this by reference variable is known as testf. Assigning a new string to testf actually assigns the string to h$ directly.

When a Function is used for a Windows API callback make sure the return data type is a primary numeric type (Byte, Word, Long, Int64, Single, Double), otherwise the stack will be overwritten.

Sub And FunctionVar parameters
For compatibility reasons GFA-BASIC 32 includes the Sub and FunctionVar statements. FunctionVar is compatible with VB’s Function; arguments are passed by reference by default and without a data type the Variant type is assumed. The same is true for Sub, without a ByVal or ByRef keyword the default is by reference, an implicit by reference. When a datatype is missing the Variant type is the default. Examples:

Sub test(vnt)        ' implicit ByRef, Variant datatype
  vnt = "new Value"  ' do not write to parameter

FunctionVar tfv(a As String) ' As Variant
  tfv = "new value" + s

Although ByRef is implied the rules aren’t as strict as with an explicit ByRef. When ByRef is included only actual variables can be passed, without the ByRef keyword the Sub and FunctionVar also accept literal values and types that don’t match. The following calls are allowed:

Local vnt As Variant, s As String
test(7)     ' 7 is assigned to a local Variant first
test(vnt)   ' vnt variable is passed by ref
test(s)     ' types don't match
vnt = tfv(7)' 7 is assigned to string first

Note that the literal value 7 is passed to the by reference parameter vnt in test. The compiler won’t complain since an implicit by reference does not have to reference an actual application varaible. When the Sub test(vnt) and FunctionVar tfv() include a ByRef keyword explicitly the compiler will check the argument’s type against the parameter’s type and will complain if they don’t match. An explicit ByRef declaration only accepts actual variables of the same type as the argument. Both, the variable that is passed as the argument, and the by reference parameter must be of the same type, like this

Dim s As String
CallByRef s     ' must be a variable
Debug s         ' = “new value”
Sub CallByRef(ByRef p As String)
  ' only accepts String variables as arguments
  p = "new value"

Limits to the use of parameters
So, there is a subtle difference between the default, an implicit ByRef and an explicit ByRef in a Sub and FunctionVar. In both cases only a variable can be passed by reference properly. With an implicit by reference an actual variable can be passed only when the types match. In all other circumstances the argument is first copied to a hidden local variable of the same datatype as the parameter and then the hidden variable is passed by reference. In the example above, the literal value 7 is first copied to a temporary hidden Variant variable and then the temporary variable is passed by reference. The same is true for the third call: test(s). Since the data types don’t match the string s is first copied to a temporary Variant variable which in turn is then passed to test(). The temporary hidden variable is immediately destroyed after returning from the Sub or FunctionVar.

Now lets look at it from the Sub’s point of view. Although by reference is implied the Sub doesn’t know what kind of variable is actually passed. It might be an actual variable, but it might also be a reference to a temporary local variable. Therefore Sub and FunctionVar cannot return a value using an implicit by reference parameter. In case a temporary variable is passed, it will be destroyed immediately after returning from the Sub. Only when a parameter is declared using the ByRef keyword explicitly is the Sub guaranteed to receive an actual variable.

Note An implicit ByRef parameter cannot be used as a local variable as can with ByVal parameters. The Sub doesn’t know whether the parameter is a reference to a temporary variable or to an actual variable. In case the parameter references an actual application variable the Sub might very well overwrite the contents of that variable.

04 March 2018

GfaWin23.Ocx Update 2.34

Another three bugs are fixed in this version. The SetPrinterByName needed maintenance because of the always growing need of memory with newer versions of Windows. The command failed with some printers that returned a lot of information in the GetPrinter() API. GFA-BASIC did not reserve enough memory and caused a buffer overrun. The EOF() function now also works with inline files, the ones that are stored in the :Files section of the program. The ListView.GetFirstVisible property now returns a ListItem.

About Version-numbering
The update gets FileVersion 2.34.1803 and still belongs to GFA-BASIC product version 2.3. The Build number now shows the year and month of the release. This is the first update with the new version structure. The DllVersion$ is 2.34 Build 1803 and indicates a release date of March 2018. A DLLVERSION structure always uses a 3 part format, major, minor and build. The VERSIONINFO resource on the other hand uses 4 part format. The GfaWin23.Ocx VERSIONINFO structure == 2.34.1803.0 and leaves the last part unused. This is the version showed in the File Properties dialog:

The ocx extension indicates a DLL with OCX-controls that are described using a type library. In contrast with the purpose of the extension, the OCX controls are not publically registered and are only available within in GFA-BASIC 32.

All OLE classes defined in the GfaWin23.ocx are private to the GFA-BASIC 32 application, each OLE class is implemented with its own command. This leaves no room for dynamically loading of other COM classes and automatic use of their interfaces (like VB). Third party COM classes can only be used when they implement a dual interface so they are accessible through a dispatch identifier. In GFA-BASIC these dual interface classes are supported through the use of CreateObject and the Object.property syntax. Unfortunately, calling an interface member (property/method) requires a two step process. First the caller must ask the server for an ID number and then use that number to actually invoke the member (property/method). Each dot operator requires this two step process and accessing dual interface members can cost quite some time when a command consists of many dots. For instance something like this: Object.List.Items(n).Text requires 6 calls to the COM-class provider. GFA-BASIC 32 provides a hidden optimization for automation objects created with CreateObject(). It caches all IDs in a hash-table the first time they are used. The next time a property/method is used it is looked up in the hash-table which is considerably faster.

24 February 2018

Non-bugs (1)– Events in a Form with controls

In the past years quite some (alleged) bugs have been reported. These reports range from editor-problems to compiler-errors to runtime bugs. Many of them have been addressed in a blog post or by a fix in an update in the GfaWin32.exe or GfaWin23.ocx binary. Still, there are issues that need some kind of an answer. In this series I’ll look at some of them.

Alleged Bug: A (single) button on a Form eats the form’s keystrokes.

This problem occurs when old-style GFA-BASIC programming is mixed with OLE Container/Control style of programming. Older GFA-BASIC (16-bit) programs often respond to messages in a message loop using GetEvent/PeekEvent. All messages are retrieved in a single place and dispatched based on a GB specific ID-value in MENU(1) or a Windows message value in MENU(11) or its alias _Mess. This is no problem as long as there are no controls on the window. 

For instance, the well-known OpenW command behaves the same as in older GFA-BASICs, that is until a control is created in its client area. When the window contains a control (OCX) it starts behaving as an OLE Control Container according COM-specifications. In GB32 the concept of simple controls no longer exists. Controls are now OLE controls wrapped in COM objects and behave according strict OLE-rules. One of the consequences is the change in keyboard handling. Now, the focus can no longer be set to the client area of the window, because the client area is now a control container site and has COM defined responsibilities; “An OLE Control Container implements keyboard handling by calling specific methods on a predefined COM interface that the controls supports.” In practice this means that all keyboard messages are send to controls and only controls can have the focus. This makes sense because an OLE control containers must support default and cancel button handling, mnemonic handling, and tab handling, including maintaining tab order.

As an example a snippet that won’t work because the old-style of message handling no longer works. The following is ‘impossible’, because the OpenW 1 is a Form (control container) with a single button OCX. All keyboard messages are (as should be) dispatched to the currently active control, which is - in all circumstances - the single button.

OpenW Center # 1
Ocx Command cmd1 = "Click", 200, 100, 190, 26
  // ~SetFocus(Win(1))  ' This doesn't help
  Select _Mess
    Print "WM_KEYDOWN"
  Case WM_CHAR
    Print "WM_CHAR"
Until Me Is Nothing

Because the button is the only OCX on the Form it will regain focus over and over, how hard you try to shift it back to the window.
If you like a Form to respond to keyboard events don’t use controls or use a separate child Form Ocx next to the other controls.

A Bug: A button on a ChildW form eats the form’s mouse clicks.

In the same situation as above, a window with a button (or a toolbar) the Form’s mouse clicks might be passed on to the button or toolbar. The bug presents itself in ChildW forms in a MDI-application, not in single window.

08 February 2018

A Manifest does not guarantee Visual Styles

This is part 2 of the problems that arise when using manifest files. In the first part: Update doesn't load manifest file we saw how to force Windows to load the common controls version 6, rather than to default to version 5. A manifest could be included as a resource, but it can also be an external file in the same directory as the executable. External manifest files do overrule the embedded manifest resource. To be sure the Windows loader recognizes the manifest file – for GfaWin32.exe - the timestamp of the exe must be newer than the manifest’s last modified date.

But there is more. A correct manifest file and correct dates do not guarantee that version 6 is actually loaded. For instance the following must be taken into account

  • If there are spaces in the .exe name (e.g. the exe is called “this is executable.exe”), the manifest file (“this is executable.exe.manifest”) would not work – Common Controls were not displayed correctly; however, if the spaces are replaced with underscores in both files (“this_is_executable.exe” and “this_is_executable.exe.manifest”) they did.
  • An exe seems to respond better when a manifest is embedded as a resource than as a stand-alone file. With an embedded resource you have some sort of guarantee that the manifest is applied and that the common control dll is loaded.

How are controls painted?
The comctrl32.dll – both version 5 and 6 - is now responsible for the drawing of all controls, even the standard ones that used to come in user32.dll. This makes it easier to paint the controls in a  unified style, where the controls from version 6 are painted according the current theme selected by the user. Common control version 5 is not affected by the theming, all controls are painted in a default style coming with a particular Windows OS. So, even without version 6 controls slightly differ from one Windows version to the other.

When the controls from version comctrl32 version 6 are used controls can look different on every other user’s screen. The developer hasn’t much to say in this. Borders, 3D effect, text-color, and background color can all be changed by the user by selecting a new theme. After selecting a new theme the new style takes effect immediately and the look of the controls change accordingly. What’s left for the developer is defining which parts of the controls are to be used. For instance, a Command button can have a basFlat or basThreeD style, but how the resulting control looks like can not determined before hand. Some themes draw the flat and 3D style exactly the same.

Older Windows allow disabling Visual Styles
In older Windows versions the user (or system manager) can overrule the use of common controls version 6. Especially with Windows 7, Vista and XP the user can disable the Visual Styles completely in Control Panel, or only for the particular application in the Properties – Compatibility tab of the executable. So, even when your application depends on version 6, an older OS might allow to disable certain features.
Starting with Windows 8 this is no longer the case; the theming is applied to every GUI element and the user cannot disable the theming.

So, for older Windows versions it is possible that a compiled and manifested exe executes without applying theming. Although your program does load comctrl32.dll v6 it might look old-school style (but hey, it is what the user wants). You don’t know what is actually painted in this situation. In fact, with all these different OS versions and personalization settings you can not predict at all how your program will look like exactly.

Recommendations for using controls

  • Use the newer common control version 6, this ensures painting according the default theme of each new Windows update and makes your app look more up-to-date.
  • Use default settings for all controls, don’t change text color or background color, these parts are controlled by the theme.
  • Add a resource manifest to the compiled exe, since using a manifest file might give you trouble.

Note Despite these recommendation GfaWin32.exe comes with a manifest file so that users can delete the manifest and use old-style controls for their legacy programs. Do remember that users can add a manifest file to your compiled exe themselves and thereby change the overall look of your app. 

Form Editor - Differences in painting OCX
GfaWin32.exe comes with a manifest file (stripped to only load common controls version 6) and the controls are displayed in the themed style see the previous post. The IDE itself uses pure API functions to create controls, but the Form Editor behaves quite different. The Form Editor uses specific COM design-time interaction between the host (Form) and the OCX controls. The runtime GfaWin23.ocx provides two different code-paths for communication between the host (Form) and OCX controls: a run-time and a design-time handling. There might be a discrepancy between these modes. The only way you can tell if an OCX behaves as expected is by checking the result by running (F5) the program. Then test it as a compiled exe. (You can use Launch Exe from the Project menu.) There should be no reason why a program RUN in the IDE behaves different from a stand-alone exe. The exact same runtime code is executed.

Note - There is a little catch. There is a flag in the runtime (gfawin23.ocx) that is set when the IDE is active and theoretically it is possible that the runtime behaves somewhat different. In the huge COM-related disassembly it is hard to determine what it actually does. Only a stand-alone exe that gets all the attention from the runtime without keeping an eye on the IDE. This might explain the discrepancies people noticed.

Is Themes Enabled
When problems occur it is possible to inquire about the theming state of the executable. First of all you need to check whether the program is actually using common controls version 6. This is accomplished by testing CommCtlVersion  property of the Screen Object:

If Screen.CommCtlVersion >= 6
  MsgBox "Common controls version 6 loaded."

If the required common controls version is loaded we also need to know if the application is actually themed (on older Windows). The controls need to be painted using the current theme. For this purpose there are some additional APIs located in uxtheme.dll (which comes with Windows XP and later). Two of them seem to provide information about the current theming state of the application, in particular IsThemeActive() and IsAppThemed(). However they come with limitations.

  • IsThemeActive() returns TRUE when Visual Styles are enabled for the user. It is a user setting that can be changed in Control Panel for Win 7 and lower. On Windows 8 and above this function always returns TRUE. IsThemeActive() isn’t very useful, it is included in the IsAppThemed() API.
  • IsAppThemed() checks to see if theming (Visual Styles) are on, which is the same as IsThemeActive(). In addition it checks the existence of a manifest file and also the Compatibility tab of the executable’s Properties. If all three conditions are true IsAppThemed() returns TRUE.

IsAppThemed() returns true if the theming is applied to the application. If also Screen.CommCtlVersion >= 6 (thus manifest is loaded) the application uses theming for common controls version 6.  With the following code snippet you can test the current theming state of the common controls version 6.

Declare Function IsAppThemed Lib "UxTheme.dll" () As Boolean
Trace Screen.CommCtlVersion >= 6 && IsAppThemed() ' True/False

This concludes the posts on manifests to enable themed common controls. If you have any comments or question please use the Comments section below.

For more information see https://www.codeproject.com/Articles/620045/Custom-Controls-in-Win-API-Visual-Styles