25 May 2017

New Group/Mailinglist

A new mailinglist has been created: the  GFAGroup. This works the same as the previous Mailinglist that went down at the beginning of this year.
This mailinglist/group is provided by Google and thus requires a Gmail-account. After you have signed in with Google using your Gmail-account you can join the list. Although it is a group, it works much in the same way as the previous mailinglist. Questions and answers are posted through e-mail.

12 May 2017

ANSI, UNICODE, BSTR and converting

In the previous post I discussed UNICODE versus ANSI in the ANSI-based GFA-BASIC. Basically, GB doesn’t support UNICODE because it expects 1-byte characters where strings are used. In UNICODE each character occupies 2 bytes and allows more than 256 characters. Conversion ANSI to UNICODE is ok, but conversion from UNICODE to ANSI might lead to a loss of characters with a value above 256. But there is more: Variants and BSTRs.

The introduction of COM in GB required the provision of a new data type, the Variant. The Variant is a 16-byte data type that holds data and a value that specifies the type of that data(LONG, CARD, DOUBLE, etc). A Variant can also be used to store (safe-) arrays, a specific COM array type, and BSTRs, special UNICODE strings. So to understand the String and BSTR/Variant in detail ….

How a String is stored
Because a BSTR is much like a GFA-BASIC String data type, I’ll first tell how a GB String is stored. You could skip this part if you already know.
Declaring (Dim) a String-variable introduces a name for a location. The String-variable itself requires four bytes to store a pointer to dynamically allocated memory for the characters. The declaration and assigning a location is handled by the compiler, the rest happens at runtime: assigning or initializing. When the String-variable is initialized a call to malloc() reserves memory for all its characters with an additional 5 bytes. The first 4-bytes are reserved to store the length of the string and the last byte for the null-byte (not included in the length value). After allocating and copying the characters, the address of the first character of the string is stored at the variable’s location, a 32-bits address or pointer.

Global a$       ' 32-bits location(=0) in data or stack
a$ = "GFABASIC" ' assign pointer (address) to location
l = Len(a$)     ' address <> 0 return length {address-4}
Clr a$ : a$= "" ' free memory, set locations to 0

- String in memory: [xxxx|cccccc…c|0]
- Initially, the variable is a null pointer, the contents of the variable’s location is 0.
- String variable points the address of the first character c.
- Length is stored in position address – 4, and does not include the terminating zero.

Obtaining the string’s length is a 2-step process. First the variable is tested for a non-null pointer and than the value of the preceding 4 bytes (string-address – 4) is returned.
- Clearing a string (or assigning an empty string “”) will free the allocated memory and reset the variable’s contents to 0.

BSTR in GB
GB does not provide a data type BSTR, but it provides limited support of hidden BSTRs to pass and obtain BSTR-strings to and from COM objects. GB handles the conversion and memory allocation for BSTRs, but it does not provide string-manipulation functions for BSTRs, or even BSTRs in Variants. More on this below.
BSTR variables are always temporary, hidden local variables used to communicate with COM properties/methods that take or return BSTR arguments. These hidden BSTR variables are always destroyed when leaving a subroutine. Even the Naked attribute won’t prevent the inclusion of the termination code.

BSTR strings are COM based strings. They are allocated from COM-memory and consequently the memory can be managed by both the provider of the COM-object provider and the client. That is the first difference. Next a BSTR contains UTF-16 coded wide characters, which I discussed in ANSI and UNICODE. The way COM stores a BSTR is much the same as GB stores a String variable. In fact, a BSTR is 32-bits location that stores a pointer to dynamically allocated memory with UNICODE formatted characters. The length of the BSTR is stored In front of the BSTR, again like GB’s String data type.

Use Variant for BSTR
Although, GB provides hidden support for BSTRs, the only way to get access to a BSTR is by using a Variant. The following example assigns a GB-String to a Variant. At runtime the code allocates a BSTR by calling SysAllocStringLen(0, Len(GB-String)) followed by copying the converted GB-String to the returned address. The address of the BSTR together with its data type is stored in the Variant. When the Variant variable goes out of scope, the BSTR from the Variant is released through a call to SysFreeString(address).

Dim vnt1 = "Hello"

Now it gets interesting. After GB invoked the SysAllocStringLen() COM API, it converts the ANSI string to UNICODE using a private conversion routine interspersing zero’s between the characters see ANSI and UNICODE. GB does not turn to the MultiByte*() APIs Windows provides, because GB supports ANSI characters only. In the conversion process to UNICODE no characters will be lost and the private function is extremely fast.

An optimized UNICODE conversion function
This knowledge makes it possible to obtain a UNICODE-string (not a BSTR) from a String argument through our own optimized conversion routine. Note

  • A UNICODE string is required if you want to use the Wide version APIs.
  • A UNICODE string does not have a length field in front of it. It is not a BSTR. It only specifies how much bytes a character occupies (2).
  • It’s memory is managed by the program through malloc() – no COM memory - and it ends with two null-bytes (although it seems 1 is ok as well).
  • The converted ANSI argument is placed in a String only because it is a convenient data type to store consecutive data.

The function makes use of the BSTR allocation and conversion functionality of the Variant.
(The $Export is there because it comes from a .lg32 file).

Function WStr(vnt As Variant) As String Naked ' Return UNICODEd string
  $Export Function WStr "(AnsiString) As String-UNICODE Naked"
  Dim BSTR As Register Long
  BSTR = {V:vnt + 8} ' BSTR address at offset 8
  Return StrPeek(BSTR, {BSTR - 4} Shl 1)
EndFunc

1. A function very well suited for the Naked attribute, because it does not contain local variables that contain dynamically allocated memory that would otherwise require explicit release code.

2. The argument of the function is ByVal As Variant. This forces the caller (calling code) to create a Variant and than pass it by value by pushing 16-bytes (4 DWords) on the stack. Whether the Variant is passed by value or by reference, the calling subroutine is responsible for freeing the BSTR stored in the Variant. However, ByVal is interesting because …

3. The GFABASIC-compiler provides a hidden optimization when you pass a literal string to a ByVal As Variant. A ByVal Variant requires16 bytes to push on the stack, but the UNICODE characters the Variant points to are already converted at compile time. Therefor the following call is extremely efficient:

Dim t$ = WStr("GFABASIC")

The GFA-BASIC compiler stores the literal string “GFABASIC” as a UNICODE sequence of bytes (2 per character) and does not need to allocate (COM) memory and convert at runtime. This also relieves the caller from releasing the BSTR-COM-memory, so the calling function doesn’t need to execute Variant destruction code.
Assigning a UNICODE formatted string this way, is almost as efficient as initializing a String with an ANSI literal string. It only takes a few cycles to call and execute the WStr() function.

4. The caller provides the String variable to store the return value of the function. That is the function’s ‘local variable’ WStr is silently declared in the calling subroutine. The hidden string is passed as a ByRef variable to the function. The return value (String) is directly assigned to the hidden variable. If an exception would occur in function Wstr() the termination code of the caller will release the hidden WStr string variable. (Therefor Naked is perfect for this function: it doesnot need to provide explicit release code.)

5. Inside the function you can see two more optimizations. First the local Long variable that stores the address of BSTR is a register variable; no stack memory and copying required. The other optimization is the Shl 1 expression that multiplies the length of the BSTR by 2. This results in an integer asm add eax, eax instruction, rather than a floating point multiplication. Also a significant optimization.

6. Other mathematic operations like V:vnt+8 and BSTR-4 are relative address operations and are properly compiled into indirect addressing instructions. So, no chance here to optimize.

I went in some detail to explain the function hoping you’ll find it useful. I hope to tell more about the way the compiler constructs subroutines and performs optimizations.

10 May 2017

Error free using a library

I use libraries a lot, but there are few things that make using them a bit obscure and may lead to the non-descript message "Load lg32 error: filename.lg32" in the status bar and debug window. In addition, a compiled library may produce strange runtime errors.

BUG - Runtime errors
When you run a project which includes a library it may generate strange, seemingly unrelated error messages. In particular the error "Hash Internal Error 1/2 (Version?)" pops up regularly. The reason for runtime errors inside the code of a library is a bug(!) in applying the setting for Branch Optimizations.

For a lg32 file, GFA-BASIC wants to apply the Full Optimization for Exe setting on the compiling process. However, it is never applied at all, because the code applies this setting in the wrong place, after the code is compiled ;). Consequently, the compiler switches to the trackbar/slider setting from Branch Optimizations.
This is a bug from a long time ago and it is simply never tested properly.

In general objectcode generated for a lg32 file is position independent, it differs from code generated for  EXE (and GLL files). Therefor, the lg32-generated code for the jump-tables for Switch/Case statements and On n GoSub/Call statements are wrong (this is also true for a GLL, for which I always use the default settings). 

The only setting that work flawlessly is the None setting of the slider in 'Branch Optimizations' and uncheck the 'Full Optimization' check box. 


A lg32-file has to be compiled using the default settings for Branch Optimizations.
The slider must be set to the first position (None) and the checkbox Full Optimization for Exe must be unchecked.

Note - The slider setting is applied to compiling code in memory, independent of the required output file type (EXE, LG32, or GLL). The most right position (Full) is exactly the same as checking the Full Optimization for Exe - box. This way you can test fully optimized code inside the IDE.

Note - The branch optimizations of the compiler do not lead to remarkable performance results. These days with fast CPUs and large caches performance increase is hard to provide, the only real performance increase is accomplished by using Naked procedures. Remember however, Naked procedures do not include termination code and do not allow exception handlers.

The $Library statement
The $Library statement loads a lg32 file into memory. But sometimes it cannot locate the lg32 file. The IDE code to find a lg32 file is a bit complicated. In some conditions you may omit the extension and in others you cannot. It depends on the inclusion of a path in $Library statement. For instance, you may include a relative path (relative to the current directory, mostly the g32-file directory, but not necessarily), but than the extension must be provided. It's all a bit incoherent. But there is a solution that always works correctly. That is - the library is always located properly.

Solution for load errors
This solution adds more functionality to the $Library statement and so it complements the current functionality. You must add a (new) register entry to the GFA/BASIC key in the HKEY_CURRENT_USER/Software setting. The key must be named "lg32path" and the value can contain multiple full paths separated by commas. (The value uses the same syntax a the PATH environment variable).

New key: "lg32path", REG_SZ
Value: "C:\GFA\Include, D:\GFA\MyLibs"

Have fun with lg32 file.

ANSI and UNICODE

Updated 21-05-2017: Sample code at the end of the post.

GFA-BASIC 32 only supports ANSI strings, not UNICODE… What exactly does that mean?

ANSI-strings consist of a sequence of bytes – the characters of a string – where each byte represents a character. This allows for 256 different characters because a byte can contain a value between 0 and 255. Restricting strings to bytes limits the number of possible – mostly for not western languages – characters. To allow for more characters each character in a string must somehow occupy more than one byte. In Windows, each Unicode character is encoded using UTF-16 (where UTF is an acronym for Unicode Transformation Format). UTF-16 encodes each character as 2 bytes (or 16 bits). UTF-16 is a compromise between saving space and providing ease of coding. It is used throughout Windows, including .NET and COM.

In UNICODE the lower 256 values represent the same characters as in ANSI, but they are stored as a sequence of 16-bits integers. Additional characters are represented with higher values above 256. In UNICODE the first 256 characters have the same value as in ANSI, but each character requires 2-bytes of storage. When you convert an ANSI string to UNICODE it becomes twice the size of the ANSI string.
Let’s see what this means from a GFA-BASIC perspective.

ANSI in a GB String
When you store a literal string like “GFABASIC” in a String (ANSI, 1-byte representation), the string is filled with 8 bytes of (hexadecimal) values 47 46 41 42 41 53 49 43.

a_t$ = "GFABASIC"   ' 47 46 41 42 41 53 49 43

The same string can be created by using Chr$() and populate these byte values. (A more general approach would be to use the Mk1$() function):

a_t$ = Chr($47, $46, $41, $42, $41, $53, $49, $43)
a_t$ = Mk1($47, $46, $41, $42, $41, $53, $49, $43)

GFA-BASIC’s string functions expect ANSI strings only, and by default GB only communicates with the ANSI version of the Windows API functions. With a little knowledge you can do more.

Windows APIs are UNICODE
Windows is an UNICODE system. When a Windows API takes a string as an parameter, Windows always provides two versions of the same API. It provides an API for ANSI stings and an API for UNICODE strings. To differentiate between ANSI and UNICODE respectively, the names of the API function either ends with uppercase A - for ANSI parameters - and uppercase W for the version that accepts or expects UNICODE. A typical example would be the  SetWindowText() API which comes in two flavors SetWindowTextA() and SetWindowTextW().
The GFA-BASIC’s built-in APIs are the ones that map to the functions that end with A. So the GB function SetWindowText() maps to the SetWindowTextA() function.

UNICODE in a GB String
By default, when you declare a literal string in your source code, the compiler turns the string's characters into an array of 8-bit data types, the String. You can not – in the same way - declare a literal UNICODE string. To assign a sequence of 2-byte characters you’ll need to use different methods. For instance by populating a String by hand. In the example above it only takes one change to create a UNICODE array of characters. Simply change the Mk1() function to Mk2():

u_t$ = Mk2($47, $46, $41, $42, $41, $53, $49, $43) + #0

Now each character occupies 2 bytes and has become UNICODE formatted, because it encodes each character using UTF16, interspersing zero bytes between every ASCII character, like so

u_t$ = Chr($47,0, $46,0, $41,0, $42,0, $41,0, $53,0, $49,0, $43,0) + #0

A GB String data type always adds a null-byte (only one) to zero-terminate the sequence of characters. Since the above assignments are GB controlled, the strings end with only one null-byte. UNICODE should end with two null-bytes. You should explicitly add an additional null at the end of the string to properly create a UNICODE string.

UNICODE is not BSTR
Note that we simply created a piece of memory to store characters in 2-bytes rather than in 1-byte. The String memory is allocated from the program’s global heap and this memory is only guarded by GB. Although the string contains UNICODE it is not a BSTR. A BSTR is a COM defined string type and is allocated from COM-memory. Both the client (a GB-program) and the provider/server have access to the same COM-memory.
When a string is assigned to a Variant, which supports BSTRs only, GB allocates COM string memory and converts the ANSI string to UNICODE.

Using pure UNICODE
The GFA-BASIC string-functions use a 1-byte character indexing system. However, you can overcome this limitation for 2-byte formatted strings and apply GB String-functions when you multiply the index and length parameters by 2. For instance:

u_t$ = Left(u_t$, ipos * 2) + #0
u_t$ = Mid(u_t$, ipos * 2, nBytes * 2) + #0

You can pass these UNICODE formatted strings to APIs that end with uppercase W. To introduce the wide character APIs to your code you must Declare them explicitly. For instance, this code displays u_t$ in the client area of a window.

Declare Function TextOutW Lib "gdi32.dll" Alias "TextOutW" ( _
  ByVal hdc As Handle,        // handle to DC _
  ByVal nXStart As Int,       // x-coordinate of starting position _
  ByVal nYStart As Int,       // y-coordinate of starting position _
  ByVal lpwString As Long,    // character string _
  ByVal cbString As Long      // number of characters _
  ) As Long

Form frm1
TextOutW(frm1.hDC, 1, 1, V:u_t$, Len(u_t$) / 2)

Remember one thing. Windows uses UNICODE only, including fonts. Whether you use TextOutW or TextOutA (as Text does), all output is performed using UNICODE fonts. The TextOutA first converts the text to UNICODE and than invokes TextOutW. By providing a UNICODE formatted to a W-version API only skips the conversion from ANSI. See below for an example.

Obtaining UNICODE text from Windows APIs
Since XP, all Windows APIs taking or returning a string parameter are implemented in UNICODE only. The ANSI version of these functions translate (or convert) the ANSI strings to and from UNICODE format. Well, GB only handles ANSI strings; it passes and retrieves ANSI strings to and from Windows APIs. What is the consequence of this restriction?

When an ANSI string is passed to an A – version of an API, the Windows API will convert the string to UNICODE and than invoke the W-version of that API. There is no loss of information in this conversion. All ANSI characters are converted to UNICODE by expanding the string with zero’s as explained above. The string-size is doubled, but that’s all.

The other way around is more problematic. A Windows API may return or provide a UNICODE formatted string containing non-ANSI characters, characters with a 2-byte value above 256 … When the A-version of the API is used to retrieve text, Windows will do the UNICODE-to-ANSI conversion on behalf of the A-version of that API and the characters with a higher value of 256 will be lost.
This won’t be a problem if the ANSI-based GFA-BASIC program is used in languages no other than Latin (English) alphabets. In other languages the Windows system accepts more characters and the text won’t be properly returned to the GFA-BASIC String data type.
When your program needs UNICODE input or use UNICODE strings you should explicitly declare all the required wide APIs. In addition, you might also need W replacements for the GDI text-out functions. To use the GB string functions, you should remember to multiply or divide all integer arguments with 2.

Displaying UNICODE glyph characters (updated 21-05-2017)
Windows 10 includes and uses a new graphical font: Segoe MDL2 Assets. This sample shows how to obtain the glyphs form the font icons for use in GB.
In the accessory Special Characters select Segoe MDL2 Assets and than select a graphical character. Write down the 16-bit value from the box at the bottom and assign it to a String. Here the value for the picture for saving is 0xE105.

image

Form frm1
ScaleMode = basPixels   ' by default
SetFont "Segoe MDL2 Assets"
' Display UNICODE string "GFABASIC"
Dim u_t$ = Mk2($47, $46, $41, $42, $41, $53, $49, $43) + #0
TextW 1, 1, u_t$

' Get a Picture Object from a glyph.
' Char-value from 'Special Characters' Accesorry
Dim hBmp As Handle, p As Picture
Dim size As SIZE
u_t$ = Mk2(0xE105)          ' the Save-glyph
TextW 1, 31, u_t$           ' show it
TextSizeW(Me.hDC, V:u_t$, Len(u_t$) / 2, size)
Get 1, 31, 1 + size.cx, 31 + size.cy, hBmp   ' a GDI-handle
Put 50, 1, hBmp             ' and test it
Set p = CreatePicture(hBmp, True)  ' into a Picture
PaintPicture p, 70, 1       ' and test it
Do
  Sleep
Until Me Is Nothing

Proc TextW(x As Int, y As Int, wstr As String)
  ' Assume Scalemode = basPixels, ScaleLeft=0, and ScaleTop=0
  TextOutW(Me.hDC, x, y, V:wstr, Len(wstr) / 2)
  ' If AutoRedraw == True draw on bitmap.
  If Me.hDC2 Then TextOutW(Me.hDC2, x, y, V:wstr, Len(wstr) / 2)
EndProc

Declare Function TextOutW Lib "gdi32.dll" Alias "TextOutW" ( _
  ByVal hdc As Handle,         // handle to DC _
  ByVal nXStart As Int,        // x-coordinate of starting position _
  ByVal nYStart As Int,        // y-coordinate of starting position _
  ByVal lpwString As Long,     // character string _
  ByVal cbString As Long       // number of characters _
  ) As Long
Declare Function TextSizeW Lib "gdi32.dll" Alias "GetTextExtentPoint32W" ( _
  ByVal hdc As Handle,        // handle to DC _
  ByVal lpString As Long,     // text string _
  ByVal cbString As Int,      // characters in string _
  ByRef lpSize As SIZE        // string size _
  ) As Long
Type SIZE
  - Long cx, cy
EndType

A few notes about this sample (compared to the previous version).

  1. The Segoe MDL2 Assets font is not a fixed-sized font (the LOGFONT member lfpitchAndFamily is not FIXED_PITCH). However, the glyphs in the font all have the same format. To obtain the size of a glyph-character we cannot use the ANSI GB functions TextWidth() and TextHeight(), since they cannot return the size of a 2-byte character. Therefor the inclusion of the TextSizeW() function.
  2. To conform to GB’s scaling the TextOutW function should take coordinates in the current ScaleMode and the text-output should obey the ScaleLeft and ScaleTop settings. In this sample TextW simply draws on a pixel resolution scale and relative to (0,0), located at the top-left of the client area. Note however that Get and Put actually use the current scaling. Be sure to use the same ScaleMode for both GB commands as API functions. (As long as B= basPixels (default scaling in GFA-BASIC, VB uses twips, do not confuse the both).

Finally, the return values of ScaleLeft and ScaleTop are wrong (al versions below Build 1200). Hope to update the GfaWin23.ocx as soon as possible).

02 March 2017

A Dynamic Array Library

James Gaite created an impressive extension to the standard GFA-BASIC 32 array functionality. In response to comments made in a thread on the GB32 forum, he has spent some time trying to compile some basic dynamic objects that will work within a UDT, as well as making GFA-BASIC's handling of arrays in variants a bit more flexible.

Download
The results are contained in the zipped library file containing a compiled library file (lg32) and a with a brief help file. For easy access you should copy the lg32-file into the source directory of the project you are working on. Including a library file in your project forces the IDE to search for that file in the current working directory first.
Please note that this is a work in progress, as can be seen by the version number.
Dynamic Array Library (version 0.23 Build 10) : DynamicArrays.zip

Main features (but not all):
  1. A Hash table (THash) that:
     - can be used in User-defined Types and Arrays, and can be passed ByRef to functions and subroutines;
       - can 'remember' sort preferences and add all new keys accordingly.
  2. . A Dynamic Variable-length String Array (DSArray or DSA) which:
       - can be used in User-defined Types and Arrays and be passed to all routines;
       - can 'remember' sort preferences and add all new additions accordingly;
       - has basic find, find/replace and find/remove functions.
  3. A basic Variable-length String Variable that can be used in User-defined Types.
  4. New commands and functions for Redim'ing and setting values in a dynamic array in a variant.
Please post any questions (or bugs you might discover) to this thread on the GB32 forum http://gb32.proboards.com/thread/117/dynamic-variables-arrays or as comments attached to this blog article.

20 February 2017

Tip: Using mt.exe to include a manifest

A tip from Peter Harder on how to use the mt.exe tool included with the Windows SDK.


The GFA-Basic editor can not include a *.manifest file into a created *.exe file. However, the Microsoft Windows Software Development Kit (SDK) contains a file named "mt.exe" that can
include the *.manifest file after creating the *.exe file.

I uploaded an examples that includes also the mt.exe and a batch file that shows how to call the mt.exe:

Just follow the instructions in the readme.txt.
For a complete description of mt.exe tool use Google and search for "mt.exe".

19 January 2016

Using WinHlp32 under Windows 7, 8 and 10

You can still use the older Help files in GFA-BASIC 32. Although Microsoft doesn’t support the older .hlp format and wants us to use Html-chm help files, the internet provides us with workarounds. The most famous is the package assembled by Komeil Bahmanpour, read more ....

Peter (_GBUSER_) Harder, a die-hard GB32 user, modified the help installation package of Komeil Bahmanpour so that you can install it on Windows 7,8, and 10.
You can either download it from his server or mine.

Download 1 (Peter H): WinHlp32_Installation
Download 2 (Sjouke H): WinHlp32_Installation

Just follow the instructions in the "readme.txt", because the help must be installed using Administrator rights.

28 November 2015

A Function creates a silent local variable

The documentation on Naked shows an optimized example of a windows-procedure callback. Interesting is the fact, that the example shows GB-code using a Procedure rather than a Function. The code shows how to pass a return value to the caller of WndProc.

Proc WndProc(hWnd As Handle, msg As Int, _
  wParam As Int, lParam As Int) Naked
  Local RetVal
  '... Code ...
  Asm mov eax, [RetVal]
EndProc

The example is misleading in the assumption that you need to use a Procedure rather than a Function!

The example merely shows how to use a local variable RetVal to return a value to the caller of the subroutine, which in this particular case is the Windows OS. The assembler instruction mov eax, [RetVal] copies the value to return from the local variable in the eax-register. Returning a 32-bit value through eax, and 64-bit values through eax and edx, is conform the Windows StdCall convention. The caller of the subroutine knows from this convention that any return value (32-bits) is placed in eax. The caller may examine the eax register and can respond accordingly.

A Function is easier and safer
Most of us won’t use a Procedure when creating a callback. Instead, we will use a Function so that the need for an explicit Asm command disappears. With Function the GB-compiler generates the code to return a value. Because the compiler takes into account the Function’s data type, it generates code to put values into the correct registers. Maybe, in case of returning a simple Long value, putting a value into the eax register isn’t very hard, but when it comes to other types, like floating-point values or even Currency values, everything becomes more complicated.
For these reasons you will always turn to the use of Functions, you do not want to fiddle around with assembler instructions just to return some value.

Procedure and Function are not so different
On a pure theoretical level something interesting can be deduced. When a Function and a Procedure-returning-a-value are compared they actually are quite the same. For a Function the compiler silently inserts a local variable of the function’s data type and name. To return a value the Return value (or Exit Func) statement copies the value into the local variable - before it copies the value to the proper register(s). The function behaves the same as a procedure with an explicit local variable.
Consequently, a Function and a Procedure-returning-a-value are equally efficient, also when Naked is used. Both need a local variable to temporarily save a return value, which is later copied to the appropriate register(s). On the other hand the Procedure type doesn’t need a local variable to store a return value. Everything can be handled by registers using assembler. When using Naked the Sub or Procedure type can be used to create highly optimized code using assembler.

Function and Naked
When performance is a concern (hardly these days so it seems), you should realize that the compiler always silently inserts a local variable when it compiles a Function statement, even when the function doesn’t declare local variables it self. The compiler generates VB-compatible code for a Function and VB uses a local variable to return a value from a function. This leads to a small decrease in performance, because he compiler has to set up prologue and epilogue code for the Function. Naked has does not omit prologue and epilogue code generation, because of the use of a silent, hidden silent local variable to be stored in the stack.

When a Sub or Procedure doesn’t use local variables and parameters Naked optimizes by omitting prologue and epilogue code. For optimization reasons it might be useful to use a Naked Procedure and return a value using assembler. A Procedure does not add an additional local variable and lends itself better for using assembler or for creating optimized code.

Closing a child process

This is part 3 of a series on launching a Win32 application from GFA-BASIC 32. In the first part I discussed the WinExec based GB32 functions WinExec and Exec. Another WinExec based command is the Shell command spawning COMMAND.COM and is used to execute MSDOS commands, mainly. All WinExec based commands do not return any value identifying the application. The return value of WinExec based processes is not a handle of process ID and the child process is not accessible through program code.

The second part discusses the System command, which does provide meaningful return values. When used as a function, System() returns an Int64 value containing the process-handle and the process ID; the values by which a process is identified. The System command returns these values in explicitly provided by-reference variables. Both values can be used subsequent API functions, from which some accept a process handle and others a process ID.

One of these functions that accepts a process handle is TerminateProcess, which forces a process to exit. Actually this command brings the process to an abrupt and crashing halt, preventing some of the usual cleanup from happening. DLLs, for example, are not notified when TerminateProcess kills one of their clients. Like TerminateThread, TerminateProcess is abrupt, messy, and best avoided whenever possible or to be used as a last resort. We’ll see when and how to use in the code below that provides a GB function to cleanly close a spawned process.

Most documentation emphasizes to let the process come to a natural end. This means that the user quits the application as any other process. When synchronization is important – the application has to wait for the child process to terminate – specify the Wait keyword with the System command. GB32 makes sure the to wait for the child process to end while keeping the user-interface responsive. See the Help file for more details.

But what if you want to close the Win32 child process from your GB program? There is no command and there is also no Windows API function to accomplish this magic. However, there is a documented https://support.microsoft.com/en-us/kb/178893 that provides the solution when you have a process ID or handle. Here is the code translated to GFA-BASIC 32.

Global Const WAIT_OBJECT_0  = 0
Global Const PROCESS_TERMINATE  = 1

OpenW 1
Dim Process As Large
Process = System("Notepad.exe")
Dim hProcess As Long  = LoLarge(Process)
Dim ProcessID As Long = HiLarge(Process)

Do
  Sleep
Until Me Is Nothing

' Close nicely or terminate abruptly
CloseApp32(ProcessID, True)
' When System return processs handle
~CloseHandle(hProcess)

Function CloseApp32(pID%, Optional Force? = False) As Bool
  ' Purpose: (Forced) Shutdown a 32-Bit Process by PID
  ' Return : True success, False (0) failure
  ' See    : https:'support.microsoft.com/en-us/kb/178893

  Local Const TimeOut = 1000 'ms
  Local Handle hProc
  ' Get process handle when termination allowed
  hProc = OpenProcess(SYNCHRONIZE | PROCESS_TERMINATE, 0,  pID)
  If hProc
    ' Post WM_CLOSE to all windows whose ProcessID == pID
    ~EnumWindows(ProcAddr(CloseApp32Enum), pID)
    ' Give process a moment to release all resources
    If WaitForSingleObject(hProc, TimeOut) == WAIT_OBJECT_0
      CloseApp32 = True  ' Return Succes
    ElseIf Force?        ' Kill it the messy way?
      CloseApp32 = TerminateProcess(hProc, 0)
    EndIf
    ~CloseHandle(hProc)
  EndIf
EndFunc

Function CloseApp32Enum(hwnd As Handle, lParam As Long) As Long Naked
  Local pID As Long
  ~GetWindowThreadProcessId(hwnd, V:pID)
  If pID == lParam _
    PostMessage hwnd, WM_CLOSE, 0, 0
  Return 1
EndFunc

The CloseApp32 function terminates an autonomously running 32-bits application by posting a WM_CLOSE to all the windows owned by the process. The window procedures of the detached application need to destroy resources that may take some time to complete. When the main thread has come to an end, all DLLs and other system resources are released. This also takes time. Therefore the function waits for at most 1000 ms to check if the process has ended. With many running applications the timeout value may be too short.

One other note. If the System function return a process handle the calling application is still responsible for closing it.

22 November 2015

Execute a new process using System

In a previous blog post Execute a new detached process I discussed GB32 commands to start a new detached program, or process. After the new process is created and executing, the parent process cannot communicate with the new process. There is no relationship between the both, hence the term detached. The GB32 commands WinExec, Exec rely on the WinExec() API, which does not return a process instance handle. The only thing you can get from WinExec is some error code when starting the new process somehow failed.

But there is System
Starting a new process might involve more than simply executing it and leaving it out there to be managed by the user. Maybe your program needs to wait for the second application to quit, or you want to execute a process with elevated privileges, or maybe you just want a handle on the new process so you can manage it programmatically. All of this can be done using System, command or function, depending on your needs.

The System command/function comes in two flavors. A simple version, a function, that only takes one parameter. Then there is the less simple one, implemented also as a command and function, taking multiple arguments. When you only need the creation of new process, for example to get a handle on it, use the simple version. It only takes one parameter, the command line. The command line is a null-terminated character string that contains the filename plus optional parameters:

Large = System("filename [parameters]")

This looks much like the WinExec() – and Exec()  – function, but without the the show-window  argument (which is most of the times a ShowWindow SW_* constant to make the new process visible). The difference between System and the other functions is the meaningful return value, a 64-bit integer holding the process-handle and its process identification number. Together these values uniquely identify the new process.

Using System requires some effort
Using the System() function comes with a responsibility. The process handle must be freed using the CloseHandle(hProcess) API.  First let’s see how to use System() and how to interpret the return value:

Global Int32 hProcess   ' keep in memory
Local Int32 ProcessId   ' you may keep it
Local Large Process64   ' can be discarded
Exe$ = "process.exe", Param$ = "-options"

Process64 = System(Exe$ & " " & Param$)
hProcess  = LoLarge(Process64)
ProcessId = HiLarge(Process64)

In this scenario the System() function replaces WinExec, and Exec (and maybe ShellExec but that is a different story).

It might be preferable, because System() invokes the CreateProcess API directly, passing it Null for all parameters, except for the lpszCmdLine argument and the argument taking if the address of a PROCESS_INFORMATION structure. This structure receives the output (or return values) of CreateProcess. It is used to store/return four values; the process an thread handles, and the process-id and the thread-id. GB32 only returns the process handle and process-id through an Int64 data-type.

Taking a closer look at those handles
In pseudo GB32 code, System(cmdline$) function acts like this:

Dim pi As PROCESS_INFORMATION
If CreateProcess(0,cmdline$,0,…, V:pi)
~CloseHandle(pi.hTread)
Return MakeLarge(pi.Process.Id, pi.hProcess)
EndIf

Generally, CloseHandle API only decrements a reference count on a handle. When the reference count reaches zero, the OS knows that the object can be freed. That’s all, it is not some kind of destroy-function. Closing a handle might become your responsibility, keep these rules in mind:

  • With the function variant of System(), GB32 only closes the handle to the primary thread. It returns the process handle and you have to close it in time.
  • With the System command both handles are automatically closed - unless you explicitly ask for them as return values.

When you specify the “hProcess var” and/or “hThread var” options with the System command you are explicitly asking to obtain the handles and taking responsibility. For instance, in the following code System does not call CloseHandle on the handles:

Global Handle hProcess
System FileName$, hProcess hProcess, hThread hThread

When, one way or the other, the program gains ownership of a handle, it is also responsible for closing it. The application should close a handle when it isn’t necessary anymore. A good place for a call to CloseHandle could be in your exit code, after the main message loop in the main section of your program:

Do
  Sleep
Until Me Is Nothing
~CloseHandle(hProcess)
' End of Program

Termination and usage count
CreateProcess creates two objects identified using handles: a handle to a thread object and a handle to a process object. These objects are given an initial usage count of one. Then, just before CreateProcess returns, the function opens both the process and the thread object and increments the usage count again to two. It then places handles and IDs in the members of the PROCESS_INFORMATION structure. In the end, CreateProcess has set the usage count for both objects to two. In the end, the handles must be decremented twice to reach zero.

Before the OS can completely free the process, it must - of course – terminate. This will decrement the usage count with one to one. To reach a zero usage count, the parent process must also call CloseHandle. Only then the final usage count od zero is reached. 
To free the primary thread object, a two step decrement is also required. The thread’s usage count is decremented when its process terminates. (During process termination the thread’s handle is closed). But this leaves the thread with a count of one. Because the System command already decremented the usage count of the thread, it can be freed as well.

The reason to immediately close the primary thread’s handle must be found in the OS. It has something to do with multiple creations of the same process. I don’t know why, but it is heavily recommended to proceed this way. The process handle on the other hand can be kept to be used in future API calls that require a process-handle.

In the next post I’ll discuss the termination of child process.

17 November 2015

Execute a new detached process

What if you want to execute another program (process) from a GFA-BASIC 32 program? GB32 offers quite some commands and functions to do so. However, every new process is eventually started by calling the CreateProcess() API function. All those functions, including Windows API functions like ShellExecute(), WinExec() invoke the CreateProcess() API, which is declared as follows:

BOOL CreateProcess(
   LPCTSTR lpszImageName,              // image file (.EXE) name
   LPCTSTR lpszCmdLine,                // command line for new process
   LPSECURITY_ATTRIBUTES lpsaProcess,  // how process will be shared
   LPSECURITY_ATTRIBUTES lpsaThread,   // how new thread will be shared
   BOOL bInheritHandles,               // TRUE to inherit handles
   DWORD fdwCreate,                    // creation flags
   LPVOID lpvEnvironment,              // new environment (default = NULL)
   LPTSTR lpszCurrentDir,              // name of new current directory
   LPSTARTUPINFO lpStartupInfo,        // gives info about new process
   LPPROCESS_INFORMATION lpProcInfo )  // receives info about new process

WinExec - Command and Function
The CreateProcess function’s parameters permit precise control over the new process’s starting conditions. Because of these big number of parameters and options, some commands provide a simplified execution of CreateProcess. The WinExec() API already invokes CreateProcess with a lot of default values to create a detached process. WinExec() only expects lpszCmdLine and a show-window value for the STARTUPINFO.dwFlags member. All other values are set to zero or other default values. The new process is not a child process; after it is started there is no elation between the parent and the child process whatsoever.
WinExec() is a GB32 built-in API function, but it is not used. Instead GB32 uses its own WinExec function and it is defined as:

[Long =] WinExec(CmdLine$, CmdShow%)             

The CmdLine$ is a null-terminated character string that contains the command line (file name plus optional parameters) for the application to be executed. WinExec passes parameters to the CreateProcess function in such a way that it is compatible to the 16-bits version of Windows. The return value is greater than 31 if the function succeeds. Otherwise it returns an error value

Value Meaning
0 The system is out of memory or resources.
ERROR_BAD_FORMAT (1) The .exe file is invalid.
ERROR_FILE_NOT_FOUND (2) The specified file was not found.
ERROR_PATH_NOT_FOUND(3) The specified path was not found.

The 32-bit WinExec() does not return an instance handle as the 16-bit WinExec() does. The return value can only be used to inspect an error value. Therefore, GB32 also defines a command variant: WinExec CmdLine$, CmdShow%.
Both versions - function and command - are linked to the same runtime function. Obviously, the return value can only be obtained with the function variant.

Exec – Command and Function
The WinExec takes two parameters, a command line argument and a show-window argument. Most of the times the second argument will be a default value of SW_SHOW or SW_SHOWNORMAL. The GB32 command Exec takes advantage of this behavior and allows the command-line parameter CmdLine$ to be split into a filename (path included) and options part. Although this might be useful for the developer, the command itself concatenates the parts before WinExec() API is invoked. The SW_SHOW is automatically used.

[Long =] Exec(file$, parameters$)   

ShellExec – Command and Function
All processes are started using CreateProcess. It depends on the function which parameters are set and to what value. The GB32 ShellExec command/function invokes the ShellExecute() API, which sets and fills another set of CreateProcess’ arguments. Where the Exec and WinExec commands allow a simple start of a named exe program, the ShellExecute() function allows to start a process by a registered extension. Depending on the version used, ShellExec allows a more extended set of parameters for CreateProcess. In itself is ShellExec also a simplified form of invoking CreateProcess. See the help for more information on the parameters that can be used. For a comparison with WinExec and Exec we’ll only look at the variant that takes two parameters and performs exactly like these GB32 functions:

[Long = ] ShellExec file$ [, parameters$]

It doesn’t matter which GB32 command/function you use to spawn a new detached process. They all invoke the CreateProcess API. The return values of the function do not provide some handle to the new process, only an error value (if less than 32). Using Exec is tempting because it provides the easiest interface where the file and parameters are split.

GFA-BASIC 32 does provide a function that bypasses the above mentioned shortcuts to CreateProcess(): the System function/command. I will discuss this briefly in a next post.

18 October 2015

The Non-existing MenuItems Collection

MenuItems is a collection representing all the menu items contained in a Form’s menu. According to COM/VB rules the next phrase should be true:
calling Add on this collection adds an item to the menu and returns a MenuItem object”.
However, there is no actual MenuItems collection you can use in GB32. The MenuItems data type does exist, but contains nothing more than empty stub functions.

Logically, the MenuItems dependent collection (it exists only when a menu-bar is present) should be used to manage the menu items. But in Dependent Collections I showed you why GB32 considers an in-between collection redundant and overkill. Generally, the collection’s implementation does nothing more than reroute all the collection’s properties and methods to the required item object (by key or index). In the case of the MenuItems collection, well GB32 skips the collection entirely.

Where do they come from
GB32 creates a MenuItem object for a menu-entry when it parses the one-dimensional string array containing the menu-entries for a window.

Menu menuEntries$()

Since the beginning, this how a GFA-BASIC menu-bar is created. Fill an array with string elements and let GB create and assign a menu-bar to the current active window. When the window is destroyed, the MenuItem objects are destroyed as well. The only other way to release the MenuItem-objects associated with a menu-bar of a Form is by using Menu Kill.

Now let’s see how to manage the menu-bar entries without the MenuItems collection. This process is not VB-compatible, as VB prescribes the use of a collection. As you probably know, dependent collections are used to iterate over the items they contain, to add new items, to remove items, etcetera. To provide these facilities, the MenuItems collection would provide the following properties and methods:

Property or method Description
Add method Add menu-items to the collection.
Count property Return the number of menu-items in the collection. Read-only.
Item method Return a menu item, by index (or by key).
Remove method Delete a menu.item from the collection, by index (or by key).
Clear Remove and delete all menu-items

Since GB32 doesn’t implement the MenuItems collection, these properties and methods aren’t available. GFA-BASIC 32 does not provide a collection to manage menu-items, but it does however provide MenuItem objects to allow access to single menu-entries. The Form supports a property that returns a MenuItem object. The property takes one parameter, a Long MenuID-value. The value must represent the menu-ID of the menu-entry, which is either the index of an element of the string-array or a designated ordinal (specified in the string element that specifies the menu-item text).

Dim mnuItem As MenuItem
Set mnuItem = frm1.MenuItem(1)

The MenuItem-objects are created at run-time when GFA-BASIC executes the command Menu $(). The ordinal ID-value is assigned both to the MenuItem object as well as to the Windows API functions. The ID-value can not be changed later on. Also, any changes made to a menu-entry using API functions will not be reflected in the MenuItem object. So, all changes of the menu-entries are to be done using a MenuItem object.

03 October 2015

Dependent Collection Objects (Non-Creatable)

A dependent collection object can only exist in the context of a parent COM object.

Often we find those collections as part of an OCX control. COM-controls that support dependent collections include the TreeView Ocx, ListView Ocx, ListBox OCX, Toolbar Ocx, etc. These Ocx-es provide run-time access to their functionality through Dependent Collection Objects. For example, the TreeView Ocx control has a Nodes collection of Node objects that represent the items in the hierarchy the Tree-View control displays. Tree-View Node items are appended or inserted by using the Add method of the Nodes collection. For a List-View control you’ll use the ListItems to add ListItem objects. A dependent collection doesn’t exist until an Ocx is created and are – in COM terms - Classes That Are Not Creatable.

These collections are also provided as a means to iterate over the items of a control. This could concern a collection of all items or a collection of selected items. The collection helps in quickly locating a specific item using a key or an index. Therefor each collection object comes with properties and methods you can use to insert, delete, and retrieve the items in the collection.

Property or method Description
Add method Add items to the collection.
Count property Return the number of items in the collection. Read-only.
Item method Return an item, by index or by key.
Remove method Delete an item from the collection, by index or by key.

I won’t go into the details about the basic services of adding, deleting, and retrieving from a collection. Suffice to say, that the methods and properties depend on keys (type String) and/or indexes. An index is a Long between one (1) and the number of items in the collection (Count). The Add method determines whether or not you associate a key with an item, otherwise the collection items are only accessible through their index. A typical statement to add a Node object, representing the first position, to a TreeView Ocx is:

Form frm1
Ocx TreeView tv1 = "", 10, 10, 230, 200

Dim nodes As Nodes, node As Node

' Add an item through Nodes Collection
tv1.Nodes.Add , , , "Item #1"

' Use Dependent Collection
Set nodes = tv1.Nodes
nodes.Add , , , "Item #2"

' These are exactly the same
Set node = nodes.Item(1)
Set node = nodes(1)   // .Item is hidden

'Using With when adding multiple nodes:
With nodes
  .Add 1, tvwChild , , "SubItem1/1"
  .Add 1, tvwChild , , "SubItem1/2"
  ' ... etc.

  .Item(1).Expanded = True
EndWith

' Additional GB32 - optimizing ways
tv1.AddItem 2, tvwChild, , "SubItem2/1"
tv1.Add 2, tvwChild , , "SubItem2/2"

' GB32 Shortcut Property
tv1(2).Expanded = True

For Each node In tv1  // rather than tv1.Nodes
  Debug.Print node.Text
Next

Do
  Sleep
Until IsNothing(Me)

In fact, these intermediate dependent collections insert additional code and slow down execution speed. These collections are introduced by VB to provide a systematic way of managing items of an OCX. In the end they only consume code and time.  To overcome these disadvantages, GFA-BASIC provides collection properties and methods to the OCX control directly. This concept makes the use of the dependent collections redundant. Even the iteration properties used with For Each is simplified, see listing above.
The following collection properties and methods have been added to the OCX-es:

Property or method Description of shortcuts
Ocx.AddItem or Ocx.Add method Add items to the collection of the Ocx.
Ocx.Count property Return the number of items in the collection. Read-only.
Ocx.Item method Return an item, by index or by key.
Ocx.Remove method Delete an item from the collection, by index or by key.
Ocx.Clear method Clears the entire collection

Using these shortcuts makes the intermediate collections redundant. The OCX-es that benefit are:

OCX Available Dependent Collections Shortcut To
TreeView Nodes Nodes
ListView ListItems, ColumnHeaders, SelectedItems, CheckedItems, Icons, SmallIcons ListItems
ToolBar Buttons Buttons
StatusBar Panels Panels
TabStrip Tabs Tabs
ImageList ListImages ListImages

Whenever you operate on an OCX-item, first check if the collection-properties are implemented onto to OCX directly.

There is one more dependent collection; the MenuItems collection of the Form Object. In the next post I’ll  show you how to handle MenuItems.

30 July 2015

Tip: How to create short jmp

When doing some assembler I noticed an important optimization setting for jumps. To force the compiler to generate short jumps (short jmp, short jz, short jnz) the Branch optimization setting must be >= 1.

GBPropertiesBranch

A short jump requires 2 bytes of opcode instructions. Otherwise a jump takes 6 bytes. Short jumps are only generates when the target is within 127 bytes offset, of course.

27 June 2014

Passing default ByRef parameters to a Sub, a flaw?

In my opinion the GFA-BASIC’s Sub procedure type is an anomaly. When possible you should use Procedure instead. Oh, and when you are too lazy to type Procedure than just use its abbreviation Proc. A Procedure takes all arguments by value by default and is completely backwards compatible to previous GFA-BASIC versions.

On the other hand, if you are stubborn enough and want to use Sub anyway, make sure you use ByRef and ByVal explicitly with all parameters. Do not expect the default ByRef passing will work, because it won’t. We discussed this earlier in The Sub-ByRef flaw. This posting ended with the quote “Why this is happening is unclear, but it is easily repaired by using ByRef explicitly in Sub headings.” I was never convinced this was a real bug and I always assumed a reasonable explanation would come up some day. And it did ….

VB is the flaw; ByRef turns into ByVal
Let us consider the default Sub by reference implementation in VB, VBA, and VBScript. An argument is passed by reference only when the caller uses a very specific syntax. Only when the program executes the subroutine without parentheses the sub gets a reference to the actual argument. When the programmer puts the arguments between parentheses the arguments are passed by value. The next sample demonstrates how and when VB(A/Script) executes the default:

' Normal VB call of a Sub
' passing a reference to Hello$
SPrintStr Hello$ Sub SPrintStr(sArg As String)

When VB encounters a call with arguments inside parentheses, the arguments are evaluated first. In other words, before something is put on the stack the expression inside the parentheses is evaluated (executed, calculated, etc) and a local copy of the variable is assigned the outcome of the evaluation. The local copy is passed by reference to the subroutine, which than becomes a by value passing of the original parameter. Using the parentheses-syntax in the next example will pass the address of a local variable to the Sub SPrintStr.

' VB call of a Sub using ()
’ making and passing a copy of Hello$
SPrintStr(Hello$)
Sub SPrintStr(sArg As String)

Conclusion VB
When VB calls the subroutine with parentheses the sub gets a reference to a copy of the argument and the default ByRef situation suddenly turns into a ByVal passing. The VB Sub and its implicit, default, by reference passing introduces performance loss and unnecessary memory consumption! Remember this when you port VB code to GB32.

You’ll find the whole story on http://blogs.msdn.com/b/ericlippert/archive/2003/09/15/52996.aspx

GFA-BASIC supports with parentheses
You can almost feel the agony FO must have felt when he found out about this VB quirk. How to maintain backward compatibility to previous GFA-BASIC versions and VB? In GB we are used to place parameters between parentheses. Should we suffer from this VB anomaly as well? Either way, FO decided to support the Sub implementation with parentheses only. Passing arguments without parentheses is equal to passing them with parentheses.

Although the Sub-ByRef problem is much clearer now, the GB32 implementation is still a bit ambiguous. Because GB32 mimics the call with parentheses only the subroutine always receives a by-value argument to a default ByRef parameter. You may omit the parentheses in GB32, but it still executes the version with parentheses. Only when a global variable is passed to a default by reference argument the actual variable address is put on the stack.

Conclusion GB32
Despite the ambiguity between global and local parameters, you should, as a general rule, no longer consider a default by reference parameter as [in/out] when you omit ByRef in a Sub declaration. Inside the Sub you don’t know if you are dealing with the actual variable [in/out] or with a copy [in]. Always use ByVal and ByRef explicitly in a Sub declaration.

31 May 2014

Problem with local Hash variables

GB32 does not support the destruction of local Hash variables. This is by design. Global Hash variables are released when the program terminates. This does not mean you can’t use local Hash variables, you just have to add the termination code yourself.

This behavior might be the cause of many reported memory leaking problems.

Let us look at some examples. First suppose you have a subroutine like this where a local hash variable is used to store the results of the Split regular expression command:

Proc Split_Local(ByRef t$, sep$)
  ' Declare a local Hash variable.
  Dim hs As Hash String

  ' Split creates a new hash table of String,
'
and destroys the hash allocated memory first. Split hs[] = t$, sep$ ' Explicitly erase Hash variable, because ' memory allocated by hs[] is not destroyed ' automatically when going out of scope. Hash Erase hs[] EndProc

The Dim command declares a Hash String variable hs. The variable is put on the stack. Stack memory is temporary space limited to the scope of the procedure call. A hash variable occupies 8 bytes divided in two Long types. The first Long is a pointer to heap memory. This pointer is set to the hash-table descriptor once a value is added to the table. Initially, the pointer is Null. The second Long contains the code for type of data the hash is going to store. Here the type indicates a String, so the hash is used to store String values.
When the hash variable goes out of scope, at the end of the procedure, the stack is cleared or reset to the value it was at the point where the procedure started executing. The 8 bytes reserved for the hash variable are simply discarded without freeing allocated memory first.
A hash table allocates memory dynamically. When the hash needs more memory, it is allocated automatically. The hash grows and shrinks automatically allocating and freeing heap memory on demand. When the hash variable goes out of scope the allocated memory is no longer referenced by any GB variable or GB garbage collector. This memory gets freed after the application has ended and all of the application’s memory is released to the OS.

  • A local hash variable must be freed explicitly using Hash Erase on the variable name.

The Split command clears the Hash table as well, that is when the first Long of the hash variable isn’t Null. Before Split starts splitting the string in to tokens that are to be stored in the Hash String variable, it completely erases the Hash. In fact, the Split command invokes the runtime function HASHERASE, which is also called with the Hash Erase command.

Static Hash
Now suppose you would like to use Static on a local hash variable. That would prevent unnecessary memory de-allocations in your procedure and would improve performance when the procedure is executed, wouldn’t it? Wrong. Look at the next example:

Proc Split_Static(ByRef t$, sep$)
  ' Static declares a global Hash variable
  ' with local scope.
  Static hs As Hash String
  Split hs[] = t$, sep$, 10
EndProc

Although the hash variable is declared local, a Static variable is actually a global variable. They are treated the same as other global variables, only their visibility is limited to the procedure they are declared.

Note When asked for the variable address using VarPtr(hs) the address returned is located in the global data section of the program. VarPtr() does not return a stack address, the Static variable is not stored in the stack. For the duration of the program execution the static/global hash variable hs[] can not be changed by other code than the procedure it is declared in. Because a static variable is assigned to the program’s data section, the GB32 application will release the memory it allocated. The static hs[] variable is destroyed when the application quits. When a program is executed from within the IDE all globals will be destroyed and so will be the static hs[] hash variable.

Let us consider the example where the static/global hash variable is passed to Split. Upon entry Split will destroy any entries the hash variable references, the hash variable is destroyed. Than, a new hash table is allocated and its new pointer is stored in VarPtr(hs) + 0. The contents of  VarPtr(hs) + 4 remains unchanged, because it specifies the hash data type. When Split finishes the global/static variable hs[] points to a new hash table. Access to the elements in hs[] is now limited to the code between the Split command and EndProc. Since the hash isn’t destroyed when the procedure returns, the program now carries with it allocated memory that cannot be accessed until the next time the procedure is called. And when the procedure is executed the Split command immediately destroys the hash table. Instead, you could insert a Hash Erase at the end of the procedure, but what does that give you? A local hash variable that is stored as a global variable.

If you like to read more on hash variable, go to Passing a hash variable to a subroutine

More on local scope
Automatic destruction of variables with local scope is only supported for String, Variant, Object,  BSTR and arrays. When a local array is used the compiler inserts a call to CLEARARR(). The code responsible for clearing other type of local variables – a function called CLEARMULTI located in the runtime GfaWin23.Ocx – accepts a Long integer (4-bytes) where each byte contains the number of local variables of one specific type. The Long is coded like this: BBVVOOSS. The Lo byte contains the number of String variables to clean and the high byte of the lo-word contains the number COM objects to release. The hi-word contains the number of Variants and BSTR types.

Yes BSTR types. Unfortunately GB does not provide a data type BSTR we can use to store UNICODE strings. However, all COM objects that require a string use BSTR as a parameter type. When the compiler creates code to invoke a method or property taking a COM string as an argument, the ANSI string is converted to a BSTR before the method or property is called. When necessary the BSTR is stored in a hidden local variable and should later be destroyed.

02 December 2013

Using methods and properties through IDispatch

This is all about two – more or less – undocumented features; the GFA-BASIC _DispID() function and a special dot-curly operator, the .{dispID} syntax.

An object’s layout in memory is, when using a BASIC Object variable data type:

MemObj vtable (array of function pointers)
AddrOf vtable AddrOf QueryInterface()
RefCount% AddrOf AddRef()
…. AddrOf Release()
…. AddrOf GetTypeInfoCount()
… data AddrOf GetTypeInfo()
  AddrOf GetIDsOfNames()
  AddrOf Invoke()
  AddrOf ComPropertyMethod1()
  AddrOf ComPropertyMethod2()
  …..

After the following commands the BASIC Object variable oIE holds a reference to an instance of Internet Explorer, an automation server and therefor guaranteed to support a dual interface.

Debug.Show
Dim oIE As Object
Set oIE = CreateObject("InternetExplorer.Application")
Trace Hex(Long{V:oIE})
Set oIE = Nothing

An Object variable is actually an Int32 data type and holds the address returned by CreateObject.
Initially, the Object variable is zero (or Null) and is interpreted as Nothing. The following statement does nothing more then checking if the variable oIE holds a value (is not zero):

If oIE Is Nothing Then _
  Debug.Print "No Object assigned"

To view the contents of the integer oIE we first needs its address. In GB32 you can obtain an address of a variable in several ways. You may use VarPtr(oIE), or V:oIE, or *oIE. After obtaining the variable address it must be read, or peeked, to get the contents. Reading a Long or Int32 from some address is done using Long{addr}. The Trace statement uses the Hex function to convert the value to a hexadecimal format, the usual format for a memory address.

From the previous blog post Using OLEVIEW we learned how to process the calling of a property or method through late binding. To summarize, to execute a method or a property of the object an automation client can:

  1. Call GetIDsOfNames to "look up" the DispID for the method or property.
  2. Call Invoke to execute the method or property by using the DispID to index the array of function pointers.

All BASIC languages support this behavior behind the curtains. They provide the object dot-operator to call properties and methods through late binding. For instance, when you want to check to see if Internet Explorer is visible you may call the Visible property:

If oIE.Visible == True Then _
  Debug.Print "Is visible indeed"

After compiling this code the program will execute oIE’s interface-function GetIDsOfNames to "look up" the DispID for the property. It will then generate code to call Invoke to execute the method or property by using the DispID to index the array of function pointers (vtable). Calling Invoke is a bit of a hassle and is best left to the compiler.

Now what when the server gets new functions and new names. Unfortunately, you would need to recompile and redistribute the client application before it would be able to use the new properties and methods. In order to avoid this, you could use a ‘CallByName’ function to pass the new property and method names as strings, without changing the application.

In contrast with other programming languages GFA-BASIC features a function called _DispID(). This function allows you to call the objects GetIDsOfNames function to "look up" the DispID for the method or property. When you remember the blog post on Using OLEVIEW you might have noticed that every property and method is assigned a unique ID (integer value). Using _DispId(0bject,Name$) we can obtain exactly that unique value. For instance, this will display the ID value 402 in the Debug output window.

Trace _DispID(oIE, "Visible")

Obtaining the dispId of a method or property is only useful when you can use the integer value to call the IDispatch function Invoke(). Specifically for this purpose GFA-BASIC 32 provides us with a might piece of equipment; the dot-curly operator. To call Invoke using the dispId you can simply replace the name of the property or method with ‘{dispId}’.

Global Long dispIdVisible
dispIdVisible =  _DispID(oIE, "Visible")
Trace oIE.{dispIdVisible}
Trace oIE.{402}

How does VB6 do it?
If you’re interested, you should compare this elegance to the VB6 function CallByName. This function allows you to use a string to specify a property or method at run time. The signature for the CallByName function looks like this:

Result = CallByName(Object, ProcedureName, CallType, Arguments())

The first argument to CallByName takes the name of the object that you want to act upon. The second argument, ProcedureName, takes a string containing the name of the method or property procedure to be invoked. The CallType argument takes a constant representing the type of procedure to invoke: a method (vbMethod), a property let (vbLet), a property get (vbGet), or a property set (vbSet). The final argument is optional, it takes a variant array containing any arguments to the procedure.

Conclusion
Due to the elegant syntax, GB detects how to invoke the method or property. The parameters of a property or method don’t go in a Variant array. Due to the dot-curly syntax the parameters are specified as any other property or method call. The only thing you need to do yourself is retrieving the dispID, but this is of great advantage since now you are able to store the ID to use it over and over. The CallByName() function each time has to obtain dispID for the name passed.