Click here to Skip to main content
Click here to Skip to main content
Go to top

VB6 and VBA: Creating a variable and an array that points to same data

, 10 Mar 2014
Rate this:
Please Sign up or sign in to vote.
This is a very fast way to access string (or other) data with minimum memory consuming

Introduction

The idea started after I’ve read the nice article by Sam about parsing strings into JSON objects. Sam decided to copy the string characters into an array of integers where is becomes much faster to read and parse data. I was surprised by the performance gain that we get even with the time wasted in copying data.

The problem was when we have very large string that we have to buffer. In this case we have to create an array of same size and duplicate the string data inside it. This will consume a lot of memory and time to copy data.

Solution:

My idea is to create an array that shares its data with the string. In other words the first integer of the array is exactly the first character of the string and changing any of them will change the other.

In this scenario we only need to create a small array (one or two elements) and then change its underlying length and data to point to the string length and characters.

Background

Both Visual Basic 6 and Visual Basic for Applications are built over COM structures and all of its data types are Automation data types. So most of the variables are pointers to COM structures.

Here we are interested in two data types: Strings and Arrays:

1. String:

Strings are saved as Unicode characters inside BSTR structure. We don’t need to Know about BSTR here. It is enough to know that we can get a pointer to the character sequence of the string by the famous undocumented function ‘StrPtr ()’.

2. Array:

Here the situation is more complicated, because we need to change the underlying structure of the array (both length and data pointer).

Arrays are saved in a SAFEARRAY structure which is defined as:

typedef struct tagSAFEARRAY {
    USHORT        cDims;        //The number of dimensions.
    USHORT        Features;    //Flags.
    ULONG        cbElements;    //The size of a single element.
    ULONG        cLocks;        //Number of locks.
    PVOID        pvData;        //The data.
    SAFEARRAYBOUND    rgsabound[1];    //one bound for each dimention.
} SAFEARRAY, *LPSAFEARRAY 

So when we get the safe array pointer we have to change the data pointer of the array (pvData) which is located after 12 bytes from the beginning of the SAFEARRAY structure.

We also need to change the array size which saved in ‘rgsabound’ member to reflect the size of the string. SAFEARRAYBOUND structure is defined as:

typedef struct tagSAFEARRAYBOUND {
    ULONG    cElements;    //The number of elements in the dimension.
    ULONG    lLbound;    //The lower bound of the dimension.
} SAFEARRAYBOUND, *LPSAFEARRAYBOUND;

So we have to change cElements (which is located after 16 bytes from the beginning of the SAFEARRAY structure) to reflect the number of elements we need for our array.

Note1: If you want to change the lower bound of your array you can also change lLbound after 20 bytes from the beginning of the SAFEARRAY structure.

Note2: Be careful if you want to create a multidimensional array. In this case every dimension will have a single SAFEARRAYBOUND elements stored in rgsabound.

How to point to SAFEARRAY structure:

It is easy to get a pointer to any object in VB6 or VBA by direct using of VarPtr () function. This is true for everything except for arrays!! If you try to pass an array to that function you will get an error.

To solve this problem we have to create a new declaration for the VarPtr () function that takes an array as parameter. The problem about this way is that we need to change our declaration if we changed our VB version…. The following code describes how to do that:

'For VB6 users:
Private Declare Function VarPtrArray Lib "msvbvm60.dll" Alias "VarPtr" (var () As Any) As Long
'For VB5 users:
Private Declare Function VarPtrArray Lib "msvbvm50.dll" Alias "VarPtr" (var () As Any) As Long
'For VBA users with Office 2010+:
Private Declare Function VarPtrArray Lib "VBE7" Alias "VarPtr" (var () As Any) As Long
'For VBA users with Office before 2010:
Private Declare Function VarPtrArray Lib "VBE6" Alias "VarPtr" (var () As Any) As Long 

How to apply this method:

1. Get the string:

If you have a small string, please don’t waste your time by using this complicated method:

Dim S as string, Count as long
S = “Some large string”
Count = Len(s)  

2. Create a buffer array:

We will create a very small array of integers to create the underlying SAFEARRAY data:

Dim Buffer (1) as Integer ‘Two elements array 

3. Get the SAFEARRAY pointer:

As we said before, the VarPtrArray function will return a pointer to a pointer to the SAFEARRAY structure, for that we used a CopyMemoryToAny function to get the pointer value:

Private Declare Sub CopyMemoryToAny Lib "kernel32.dll" Alias "RtlMoveMemory"(ByRef Destination As Any, ByVal Source As Long, ByVal length As Long) 
Dim pArray as Long, pSafeArray as Long
pArray = VarPtrArray(Buffer ())
CopyMemoryToAny pSafeArray, pArray, 4  

4. Backup the SAFEARRAY original data:

It is very important to back up the old values of the SAFEARRAY structure and then restore them before VB clean this variable. If we fail in this the original values will remain uncleared and we may get into memory leakage problems:

Dim pOldData as Long
CopyMemoryToAny pOldData, pSafeArray + 12, 4 

5. Change to SAFEARRAY data to the string data:

Here we have to change the data pointer of the safe array (located after 12 bytes from the beginning) and then number of elements (located after 16 bytes):

Private Declare Sub CopyAnyToMemory Lib "kernel32.dll" Alias "RtlMoveMemory"(ByVal Destination As Long, ByRef Source As Any, ByVal length As Long)
CopyAnyToMemory pSafeArray + 12, StrPtr (S), 4
CopyAnyToMemory pSafeArray + 16, Count, 4 

6. Do your work:

Now you can start manipulating, parsing or even changing string data without changing its size. Please be careful not to erase your string variable or array data. This may cause unexpected behavior.

7. Restore the original SAFEARRAY data:

Now we can finish our work by restoring the old values of SAFEARRAY structure, so when VB start to clear unused data it will clear the right data of the array:

CopyAnyToMemory pSafeArray + 12, pOldData, 4
CopyAnyToMemory pSafeArray + 16, 2, 4 

Alternatives:

We can get direct access to string data also by using the undocumented function GetMem2 which can be declared as:

‘For VB6 users
Private Declare Sub GetInteger Lib "MSVBVM60.dll" Alias "GetMem2" (ByRef Src As Any, ByRef Dst As Integer)
‘For VB5 users
Private Declare Sub GetInteger Lib "MSVBVM50.dll" Alias "GetMem2" (ByRef Src As Any, ByRef Dst As Integer) 

There are two problems about these functions:
1. They are not available for VBA users.
2. They are slower than direct access by arrays which we described earlier, because there is little overhead when calling dll imported functions.

Example:

In the attached sample we created a 20 MB string then we started to count the numeric characters inside it using three five methods:


1. By using Mid$() function (String Access).
2. By creating a buffer array of integers and copy the string data into it (Array access).
3. By using GetMem2 () function (Memory Access).
4. By creating an array of integers that point to the string data (Direct access).
5. By using a dll function written using C++ 2012 (C++ API access), this will work only from the exe file not from the IDE unless you have changed it declaration.


The performance gain against String access was really huge (30 times faster in EXE), where the speed’s gain between Array and Direct access was very small…. But actually there was a memory’s save of about 40 MB because we did not use a copy of data to parse the string. On the other hand, Memory access gave us a good performance with no memory over load and less complicated code, it is about 2-3 times slower than the last two methods. And for sure the out performance winner is C++ API access which is the fastest, but not much more than direct access method.

Caution!!! :

There is a strange behavior in this method….. When you run your application in inside the IDE it will work perfectly, but when you compile it, it will raise a ”Subscript out of range (Error 9)”, unless you compile it with removing bound array checks. You can remove it when you compile the project by pressing Option button, then go to compile tab and press Advanced Optimizations button then check Remove array bound checks.

More to do……

In this article we explained how to parse a string by direct access to its characters using an array of integers. You can also change its data if you did not change its length, for example formatting the strings to change some characters to upper case. You can also use the same concept to access any type of memory data by using arrays, For example GDI+ bitmaps data through there Lockbits function.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

EiadXP
Software Developer
Syrian Arab Republic Syrian Arab Republic
No Biography provided

Comments and Discussions

 
QuestionA nice VB6 example! PinmemberISpliter3-Mar-14 14:01 
GeneralRe: A nice VB6 example! PinprofessionalEiadXP8-Mar-14 3:47 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140922.1 | Last Updated 10 Mar 2014
Article Copyright 2014 by EiadXP
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid