Non-bugs (1)– Events in a Form with controls

In the past years quite some (alleged) bugs have been reported. These reports range from editor-problems to compiler-errors to runtime bugs. Many of them have been addressed in a blog post or by a fix in an update in the GfaWin32.exe or GfaWin23.ocx binary. Still, there are issues that need some kind of an answer. In this series I’ll look at some of them.

Alleged Bug: A (single) button on a Form eats the form’s keystrokes.

This problem occurs when old-style GFA-BASIC programming is mixed with OLE Container/Control style of programming. Older GFA-BASIC (16-bit) programs often respond to messages in a message loop using GetEvent/PeekEvent. All messages are retrieved in a single place and dispatched based on a GB specific ID-value in MENU(1) or a Windows message value in MENU(11) or its alias _Mess. This is no problem as long as there are no controls on the window. 

For instance, the well-known OpenW command behaves the same as in older GFA-BASICs, that is until a control is created in its client area. When the window contains a control (OCX) it starts behaving as an OLE Control Container according COM-specifications. In GB32 the concept of simple controls no longer exists. Controls are now OLE controls wrapped in COM objects and behave according strict OLE-rules. One of the consequences is the change in keyboard handling. Now, the focus can no longer be set to the client area of the window, because the client area is now a control container site and has COM defined responsibilities; “An OLE Control Container implements keyboard handling by calling specific methods on a predefined COM interface that the controls supports.” In practice this means that all keyboard messages are send to controls and only controls can have the focus. This makes sense because an OLE control containers must support default and cancel button handling, mnemonic handling, and tab handling, including maintaining tab order.

As an example a snippet that won’t work because the old-style of message handling no longer works. The following is ‘impossible’, because the OpenW 1 is a Form (control container) with a single button OCX. All keyboard messages are (as should be) dispatched to the currently active control, which is - in all circumstances - the single button.

OpenW Center # 1
Ocx Command cmd1 = "Click", 200, 100, 190, 26
Repeat
  DoEvents
  // ~SetFocus(Win(1))  ' This doesn't help
  Select _Mess
  Case WM_KEYDOWN
    Print "WM_KEYDOWN"
  Case WM_CHAR
    Print "WM_CHAR"
  EndSelect
Until Me Is Nothing

Because the button is the only OCX on the Form it will regain focus over and over, how hard you try to shift it back to the window.
If you like a Form to respond to keyboard events don’t use controls or use a separate child Form Ocx next to the other controls.

A Bug: A button on a ChildW form eats the form’s mouse clicks.

In the same situation as above, a window with a button (or a toolbar) the Form’s mouse clicks might be passed on to the button or toolbar. The bug presents itself in ChildW forms in a MDI-application, not in single window.

A Manifest does not guarantee Visual Styles

This is part 2 of the problems that arise when using manifest files. In the first part: Update doesn't load manifest file we saw how to force Windows to load the common controls version 6, rather than to default to version 5. A manifest could be included as a resource, but it can also be an external file in the same directory as the executable. External manifest files do overrule the embedded manifest resource. To be sure the Windows loader recognizes the manifest file – for GfaWin32.exe - the timestamp of the exe must be newer than the manifest’s last modified date.

But there is more. A correct manifest file and correct dates do not guarantee that version 6 is actually loaded. For instance the following must be taken into account

• If there are spaces in the .exe name (e.g. the exe is called “this is executable.exe”), the manifest file (“this is executable.exe.manifest”) would not work – Common Controls were not displayed correctly; however, if the spaces are replaced with underscores in both files (“this_is_executable.exe” and “this_is_executable.exe.manifest”) they did.
• An exe seems to respond better when a manifest is embedded as a resource than as a stand-alone file. With an embedded resource you have some sort of guarantee that the manifest is applied and that the common control dll is loaded.

How are controls painted?
The comctrl32.dll – both version 5 and 6 - is now responsible for the drawing of all controls, even the standard ones that used to come in user32.dll. This makes it easier to paint the controls in a  unified style, where the controls from version 6 are painted according the current theme selected by the user. Common control version 5 is not affected by the theming, all controls are painted in a default style coming with a particular Windows OS. So, even without version 6 controls slightly differ from one Windows version to the other.

When the controls from version comctrl32 version 6 are used controls can look different on every other user’s screen. The developer hasn’t much to say in this. Borders, 3D effect, text-color, and background color can all be changed by the user by selecting a new theme. After selecting a new theme the new style takes effect immediately and the look of the controls change accordingly. What’s left for the developer is defining which parts of the controls are to be used. For instance, a Command button can have a basFlat or basThreeD style, but how the resulting control looks like can not determined before hand. Some themes draw the flat and 3D style exactly the same.

Older Windows allow disabling Visual Styles
In older Windows versions the user (or system manager) can overrule the use of common controls version 6. Especially with Windows 7, Vista and XP the user can disable the Visual Styles completely in Control Panel, or only for the particular application in the Properties – Compatibility tab of the executable. So, even when your application depends on version 6, an older OS might allow to disable certain features.
Starting with Windows 8 this is no longer the case; the theming is applied to every GUI element and the user cannot disable the theming.

So, for older Windows versions it is possible that a compiled and manifested exe executes without applying theming. Although your program does load comctrl32.dll v6 it might look old-school style (but hey, it is what the user wants). You don’t know what is actually painted in this situation. In fact, with all these different OS versions and personalization settings you can not predict at all how your program will look like exactly.

Recommendations for using controls

• Use the newer common control version 6, this ensures painting according the default theme of each new Windows update and makes your app look more up-to-date.
• Use default settings for all controls, don’t change text color or background color, these parts are controlled by the theme.
• Add a resource manifest to the compiled exe, since using a manifest file might give you trouble.

Note Despite these recommendation GfaWin32.exe comes with a manifest file so that users can delete the manifest and use old-style controls for their legacy programs. Do remember that users can add a manifest file to your compiled exe themselves and thereby change the overall look of your app. 

Form Editor - Differences in painting OCX
GfaWin32.exe comes with a manifest file (stripped to only load common controls version 6) and the controls are displayed in the themed style see the previous post. The IDE itself uses pure API functions to create controls, but the Form Editor behaves quite different. The Form Editor uses specific COM design-time interaction between the host (Form) and the OCX controls. The runtime GfaWin23.ocx provides two different code-paths for communication between the host (Form) and OCX controls: a run-time and a design-time handling. There might be a discrepancy between these modes. The only way you can tell if an OCX behaves as expected is by checking the result by running (F5) the program. Then test it as a compiled exe. (You can use Launch Exe from the Project menu.) There should be no reason why a program RUN in the IDE behaves different from a stand-alone exe. The exact same runtime code is executed.

Note - There is a little catch. There is a flag in the runtime (gfawin23.ocx) that is set when the IDE is active and theoretically it is possible that the runtime behaves somewhat different. In the huge COM-related disassembly it is hard to determine what it actually does. Only a stand-alone exe that gets all the attention from the runtime without keeping an eye on the IDE. This might explain the discrepancies people noticed.

Is Themes Enabled
When problems occur it is possible to inquire about the theming state of the executable. First of all you need to check whether the program is actually using common controls version 6. This is accomplished by testing CommCtlVersion  property of the Screen Object:

If Screen.CommCtlVersion >= 6
  MsgBox "Common controls version 6 loaded."
EndIf

If the required common controls version is loaded we also need to know if the application is actually themed (on older Windows). The controls need to be painted using the current theme. For this purpose there are some additional APIs located in uxtheme.dll (which comes with Windows XP and later). Two of them seem to provide information about the current theming state of the application, in particular IsThemeActive() and IsAppThemed(). However they come with limitations.

• IsThemeActive() returns TRUE when Visual Styles are enabled for the user. It is a user setting that can be changed in Control Panel for Win 7 and lower. On Windows 8 and above this function always returns TRUE. IsThemeActive() isn’t very useful, it is included in the IsAppThemed() API.
• IsAppThemed() checks to see if theming (Visual Styles) are on, which is the same as IsThemeActive(). In addition it checks the existence of a manifest file and also the Compatibility tab of the executable’s Properties. If all three conditions are true IsAppThemed() returns TRUE.

IsAppThemed() returns true if the theming is applied to the application. If also Screen.CommCtlVersion >= 6 (thus manifest is loaded) the application uses theming for common controls version 6.  With the following code snippet you can test the current theming state of the common controls version 6.

Declare Function IsAppThemed Lib "UxTheme.dll" () As Boolean
Trace Screen.CommCtlVersion >= 6 && IsAppThemed() ' True/False

This concludes the posts on manifests to enable themed common controls. If you have any comments or question please use the Comments section below.

For more information see https://www.codeproject.com/Articles/620045/Custom-Controls-in-Win-API-Visual-Styles

Update doesn’t load manifest file

Problem: After installation of the latest update GFA-BASIC 32 shows old-style common controls. There is only one reason for this failure: the manifest file isn’t loaded.
After starting GFA-BASIC 32 the differences are clear from the tabs in the sidebar:

The left image shows the IDE without common controls 6.0 support as opposed to the right image that shows new style controls.

GB External Manifest
GFA-BASIC 32 uses an external, side-by-side, manifest. The manifest is used to enable the common controls version 6 for the GfaWin32.exe. The manifest could have been added as a resource, but not all GB32-programs support the windows common controls version 6. In general, an external manifest should be as good as a resource enabled EXE. However, modified external manifests aren't applied always properly or aren’t applied at all. Windows doesn’t always recognize a new manifest and it requires a reboot to get a new manifest loaded. But, not even that solves the problem for everybody. Some never suffer from this problem, others seem not to get rid of it. At first I couldn’t reproduce this behavior, let alone I could solve it ...

After some research I came across the following pages blog #1 and blog #2 that explain this behavior. Coupled with the bits and pieces I further read on the internet I hope to have found an answer and it is not entirely the same as described.
 
The Activation Context Cache
To improve CreateProcess performance, manifests are cached in the Activation Context Cache. Its not the manifest-file that gets cached, but a note that some EXE requires a Manifest-file. The cache stores the full path of the exe and the last modified time of the exe file. To force a reload of the manifest-file the last-modified date of the EXE must be newer than that of the manifest file.But that doesn’t always solve the problem, the cache’s entry must be removed first.

Re-enable the manifest
The binaries from the GFA-BASIC 32 Update are newer than the manifest file and oddly the controls are still displayed in the old-style. Seems the theory doesn't apply entirely, otherwise the manifest would be loaded properly. Probably the cache must be cleared first (reboot) because it contains data that prevents the loader from applying the manifest. We need to invalidate the cache’s entry for GfaWin32.exe. One way to proceed is like this.

1. Make sure the modification date of the EXE is more recent than the last-modified date of the manifest file (more about this in a minute).
2. Start GfaWin32.exe without a manifest present to invalidate (or remove) the entry in the cache that applies to the GfaWin32.exe. For example. rename the manifest to something like GfaWin32(1).exe.manifest. Old-style controls will be displayed.
3. Reboot the computer.
4. Rename the manifest to its previous name: GfaWin32.exe.manifest
5. Start GFA-BASIC 32 (and keep your fingers crossed).

Only way I got it right
This process described above might fail again, at least with me it didn’t work all the time. The only way I could make this work is by re-setting the last-modified date of the GfaWin32.exe. I had to explicitly apply the Touch # command on the GfaWin32.exe to have the manifest-file loaded and applied.

Of course, as a programmer you might like to write your own Touch utility, but you cannot Run it inside the IDE to access the GfaWin32.exe you are running from. For convenience reasons you could download this Touch tool and copy it to the GFA-BASIC’s Bin directory. Before running the tool make sure GfaWin32.exe isn’t running.

New Update Feb 2018
In Download section you can also find the new time stamped files included in Feb 2018 update zip files. The manifest file itself has changed also, it is now simple file as discussed below. Hope it fixes the problem.

Add Manifest-resource to your own EXE
When researching this problem I tried different manifest files. I found the manifest file generated from mt.exe as described in Using mt to include manifest on this blog very useful. The manifest only specifies the dependency on the common controls 6.0. The manifest is very clean and isn’t dependent on version-info and the name of the exe. You get this same manifest-file as a resource in your own EXE when you check the Add manifest resource checkbox in the compile-to-exe dialog box. You can also add it as a ‘side-by-side’ file and put it in the same directory as your exe.