 |
 | Leave me a message 
Thursday, March 25, 2004 10:18 AM
|
|
|
|
|
|
Thank you for your debugging article series.
I find them quite useful.
Rob Buldoc
|
|
|
|
|
|
|
Glad to hear it 
What type of debugging do you do and what tools do you normally use?
|
|
|
|
|
|
|
Hi Toby,
My name is albert burbea, I am a driver programmer at my company and Iahve a nasty question
I would like to limit physical memory access to Windows, using the MAXMEM flag in boot.ini, and I would like then to access this memory mapping it as (for example) a PCI card. Do you have any ideas at how can I do it?
The question is : how do I know which is the maximum physical address windows is able to access?
Many thanks
Albert
|
|
|
|
|
|
|
Hi Toby,
albert again ....
As you may remember, I am trying to hide the upper physical memory from windows and then allocate it to my driver
I succeeded to obatin the top of memory by GlobalMemoryStatus, but now I do not really know how to go on. The problem is that I am working with windriver tools suite (www.jungo.com) and they sugget to do WD_CardRegister with the phys address of the memory and the size. I may obtain 10 Kbytes of memory this way, but if I try something serious (10MBytes), the code crashes
This is waht nalyze -v returns to me:
IRQL_NOT_LESS_OR_EQUAL (a)
An attempt was made to access a pagable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If a kernel debugger is available get the stack backtrace.
Arguments:
Arg1: 00000004, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000000, value 0 = read operation, 1 = write operation
Arg4: 804e6617, address which referenced memory
Debugging Details:
------------------
***** Kernel symbols are WRONG. Please fix symbols to do analysis.
READ_ADDRESS: unable to get nt!MmPoolCodeEnd
unable to get nt!MmSpecialPoolEnd
unable to get nt!MmPagedPoolEnd
unable to get nt!MmNonPagedPoolEnd
unable to get nt!MmNonPagedPoolStart
unable to get nt!MmSpecialPoolStart
unable to get nt!MmPagedPoolStart
unable to get nt!MiSessionPoolStart
unable to get nt!MiSessionPoolEnd
unable to get nt!MmNonPagedPoolExpansionStart
unable to get nt!MmPoolCodeStart
00000004
CURRENT_IRQL: 2
FAULTING_IP:
nt!IoAllocateIrp+9f
804e6617 8b4804 mov ecx,[eax+0x4]
DEFAULT_BUCKET_ID: DRIVER_FAULT
BUGCHECK_STR: 0xA
LAST_CONTROL_TRANSFER: from 80532d2e to 804e3592
STACK_TEXT:
WARNING: Stack unwind information not available. Following frames may be wrong.
f5311f94 80532d2e 00000003 00000004 804e6617 nt!DbgBreakPointWithStatus+0x4
f5312374 804e187f 0000000a 00000004 00000002 nt!KeDeregisterBugCheckReasonCallback+0x6c7
f5312394 804e6617 badb0d00 81053000 00000000 nt!Kei386EoiHelper+0x2823
f5312424 804eb71a 014f0000 014f0000 f53124f0 nt!IoAllocateIrp+0x9f
f531245c 804e1718 00000000 014f0000 01000000 nt!PsReturnProcessNonPagedPoolQuota+0x2d6
f5312474 80565579 badb0d00 fffff000 0053be58 nt!Kei386EoiHelper+0x26bc
f53124f0 80568bcb 014f0000 00001000 00000001 nt!ProbeForWrite+0x39
f5312558 80568e52 014f0000 00001000 00000004 nt!NtFreeVirtualMemory+0x504
f5312570 bf830fd3 014f0000 00001000 00000004 nt!MmSecureVirtualMemory+0x15
f53125d8 804de7ec 040107be 00000000 00000000 win32k!EngCreateBitmap+0x1335
f5312604 7c90eb94 badb0d00 0113f4d4 f531268c nt!ZwYieldExecution+0xb78
0113f780 773e5e47 040107be 0113f7ac 00000000 ntdll!KiFastSystemCallRet
0113f80c 773e5569 000c4da0 00000002 00000000 0x773e5e47
0113f828 010276ca 000c4da0 00000002 00000000 0x773e5569
0113f85c 010078bb 00000002 00000005 0113f898 0x10276ca
0113f88c 01007808 00000006 0113f910 0113f960 0x10078bb
0113f89c 010077bc 00030046 000100c2 0113fad4 0x1007808
0113f960 01001b5e 00030042 0000004a 000100c2 0x10077bc
0113f984 77d48734 00030042 0000004a 000100c2 0x1001b5e
0113f9b0 77d48816 01001b1f 00030042 0000004a 0x77d48734
0113fa18 77d4b4c0 0009fa68 01001b1f 00030042 0x77d48816
0113fa6c 77d5e7fe 0053b980 0000004a 000100c2 0x77d4b4c0
0113fa9c 7c90eae3 0113faac 00000400 00000400 0x77d5e7fe
0113fed4 77d49402 0113ff28 00000000 00000000 ntdll!KiUserCallbackDispatcher+0x13
0113ff00 010019c1 0113ff28 00000000 00000000 0x77d49402
0113ff44 01011e8b 00000000 0113ffb4 77f74292 0x10019c1
0113ff50 77f74292 010460d8 0000005c 00090000 0x1011e8b
0113ffb4 7c80b50b 00000000 0000005c 00090000 0x77f74292
0113ffec 00000000 77f74223 0007fdbc 00000000 0x7c80b50b
FOLLOWUP_IP:
win32k!EngCreateBitmap+1335
bf830fd3 8945c8 mov [ebp-0x38],eax
FOLLOWUP_NAME: MachineOwner
SYMBOL_NAME: win32k!EngCreateBitmap+1335
MODULE_NAME: win32k
IMAGE_NAME: win32k.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 422511a2
STACK_COMMAND: kb
BUCKET_ID: WRONG_SYMBOLS
Followup: MachineOwner
---------
kd> kb
ChildEBP RetAddr Args to Child
WARNING: Stack unwind information not available. Following frames may be wrong.
f5311f94 80532d2e 00000003 00000004 804e6617 nt!DbgBreakPointWithStatus+0x4
f5312374 804e187f 0000000a 00000004 00000002 nt!KeDeregisterBugCheckReasonCallback+0x6c7
f5312394 804e6617 badb0d00 81053000 00000000 nt!Kei386EoiHelper+0x2823
f5312424 804eb71a 014f0000 014f0000 f53124f0 nt!IoAllocateIrp+0x9f
f531245c 804e1718 00000000 014f0000 01000000 nt!PsReturnProcessNonPagedPoolQuota+0x2d6
f5312474 80565579 badb0d00 fffff000 0053be58 nt!Kei386EoiHelper+0x26bc
f53124f0 80568bcb 014f0000 00001000 00000001 nt!ProbeForWrite+0x39
f5312558 80568e52 014f0000 00001000 00000004 nt!NtFreeVirtualMemory+0x504
f5312570 bf830fd3 014f0000 00001000 00000004 nt!MmSecureVirtualMemory+0x15
f53125d8 804de7ec 040107be 00000000 00000000 win32k!EngCreateBitmap+0x1335
f5312604 7c90eb94 badb0d00 0113f4d4 f531268c nt!ZwYieldExecution+0xb78
0113f780 773e5e47 040107be 0113f7ac 00000000 ntdll!KiFastSystemCallRet
0113f80c 773e5569 000c4da0 00000002 00000000 0x773e5e47
0113f828 010276ca 000c4da0 00000002 00000000 0x773e5569
0113f85c 010078bb 00000002 00000005 0113f898 0x10276ca
0113f88c 01007808 00000006 0113f910 0113f960 0x10078bb
0113f89c 010077bc 00030046 000100c2 0113fad4 0x1007808
0113f960 01001b5e 00030042 0000004a 000100c2 0x10077bc
0113f984 77d48734 00030042 0000004a 000100c2 0x1001b5e
0113f9b0 77d48816 01001b1f 00030042 0000004a 0x77d48734
kd> .reboot
(.reboot is mine)
What went wrong? and how can i bypass this ?
Thank you
Albert
PS. To make is simpler: I need to allocate several tens of megabytes of memory, and have it accessible from kernel. The memory MUST be physically contiguous. If you have any ideas, I will be more than glad to hear
you can write me at albert@gigalinx.net or albertbu@gmail.com
Thank you again
Albert
|
|
|
|
|
|
|
Your kernel symbols are wrong it says.
!symfx c:\symbols
.reload
Then do the analyze. However from this information:
Arg1: 00000004, memory referenced
FAULTING_IP:
nt!IoAllocateIrp+9f
804e6617 8b4804 mov ecx,[eax+0x4]
EAX == NULL.
From my kernel this is what the assembly looks like when I unassemble that location:
nt!MiRemovePageByColor+0x63:
804e6610 8b0485d0065580 mov eax,[nt!MmPageLocationList (805506d0)+eax*4]
804e6617 8b4804 mov ecx,[eax+0x4]
Anyway, MmAllocateContiguousMemory/MmAllocateContiguousMemorySpecifyCache can allocate phyiscally continous memory. There are other APIs that allocate this memory "safe for DMA", AllocateCommonBuffer, which do the same but ensure that the memory can be accessed by both the processor and the DMA controler at the same time. Is there really a difference? Perhaps there could be, it could even just be ensuring that the memory is allocated in a physical address range accessible by DMA or even not consuming from non-paged pool (as the previous APIs do) but from a special pool for DMA buffers. I don't know I haven't looked at those APIs to see what the difference is, could be nothing.
In anycase, you mentioned before that you are reserving the memory above what the OS thinks is the physical limit for your operations. Did you also need to access that memory from the kernel or just the device? Is this the memory you are having a problem with or is it just that you're allocating 10 megabytes of standard non-paged pool and not continous memory?
8bc7c0ec02c0e404c0cc0680f7018827ebee
|
|
|
|
|
|
|
Hi
you are really helpful, thanks
The problem I am trying to solve is as follow:
I need to allocate a HUGE contiguous buffer (> 50 MB< even 100MB sometimes), and I understood that operating system limits this. So, I decided to "hide" memory from it and try to use for my own purposes. So, how do I do this? Any ideas? I am not a device driver developer, I am using a toolkit (Jungo's windriver) that should help with this. The problem with the toolkit is that I am not neither able to allocate the memory I need nor to allocate the same amount all the time. I mean that sometimes my application tries to allocate say 16 MBytes, and succeeds, but nothing assures me that at the next run I will be able to get it.
THank you again for your time.
Albert
BTW, if you prefer to call me, or have me call you, send me an email at albert@gigalinx.net
Bye
|
|
|
|
|
|
|
Well, here is the problem.
1. You can't use the memory above what you told the Operating System there was available.
You wanted to tell the OS that the memory above a certain address didn't exist so your device could use it. This is fine if your device is the only one using it. The OS won't map those physical addresses into the page tables if it doesn't know about it. Your driver can't access the memory if it's not mapped into the page tables, unlike the hardware you can't access physical memory directly.
2. Why do you need very large buffers?
Generally what happens is that the buffer sizes are negociated and they are usually several small buffers, say 5 buffers of 64k for example. These are then used in sequence between the hardware and the device drivers. The device driver can fill up 64k of memory and let the device use it while it fills up the next, or vice versa. Applications running in user mode don't access this memory directly either, they usually issue a request to the kernel to recieve these data buffers and the driver then takes the memory and fills up the user's request.
So you should be able to allocate small physically continous amounts of memory. What is the purpose of the large buffer?
8bc7c0ec02c0e404c0cc0680f7018827ebee
|
|
|
|
|
|
|
Hi
some explanation:
we are using a standard network interface card as a frame grabber. We designed a camera with a GigE interface, and recevie the data using the NIC. The problem is that we send sometimes HUGE images (11 MBytes!) and we need it continuous. MOre, the image is preceeded or followed by data. Also, for other reasons, we need to configure the NIC in a way that it should be able to build correctly the image in memory. Of course , we may need SEVERAL buffers like this, and this is the reeason why I need such a huge buffer. I was wandering if I can map the hidden memory as a sort of a "card" in the PC, and then make reads and writes to it.
Anyway, I really need to know the base address of this hidden memory.
Again, thanks.
Albert
|
|
|
|
|
|
|
Since physical memory is linear, if Windows thinks there's 32 Meg on the system and there's really 64, then you just need to start at the physical offset of 32 megabytes. The following API can be used by devices to map their memory ranges into virtual memory:
MmMapIoSpace[^]
Mapping BUS Relative Addresses[^]
How To Map Adapter RAM into Process Address Space[^]
I would attempt to give those a read and attempt to experiment with them. I usually don't work directly with hardware so I can't tell you what the pitfalls are going to be. The other alternative, it's just an idea I had, if those don't work would be to just simply create MDL's and modify them yourself to point to your physical memory and attempt to lock them. It may or may not work but MDL's generally just describe physical pages in memory.
Here are MDL functions which you may need to use them anyway for other purposes:
MDL Functions[^]
There is something else you may want to know that different types of devices may have their own "API" for mapping device memory to physical memory. Certain drivers have a framework and most of the APIs available to them (such as video and ndis network drivers) are done through calls to functions that are basically wrappers around the system API. Some are direct call through while others do slightly different operation.
The following examples are how VideoPort drivers map AGP and Video memory to physical:
Video Port Memory Mapping[^]
AGP Memory Mapping functions[^]
This is just an FYI if you start looking around for ways to map physical to virtual memory. If you are programming for a particular driver framework you may want to check it out to see if it supports physical to virtual memory mapping. Although, again it may not be best to use it if they do something "special" for that device since you are actually mapping RAM and not device memory.
Here's another thought though, if you have a device that requires 10 Megabytes of memory to perform transfers, why not just put 10 Megabytes of memory on that device and then you would have no problems mapping it or reserving it from RAM?
8bc7c0ec02c0e404c0cc0680f7018827ebee
|
|
|
|
|
|
|
Hi,
its Albert again...
About your questions:
1. Yes, I need access from both the kernel, the device and the user
2. I need the memory to be continuous - this is because of the data we are receiving.
3. Where is the memory really beginning? I mean, there must be place for PCI boards and other system peripherals in the PC. Where the phys. memory really begins ?
THank you
Albert
|
|
|
|
|
|
|
Hi,
I wrote the Device Drivers for PCI Card in C Language.I used only I/O Mapping only.The driver is working fine.Now i want the same driver to be work in PCI Express Card.so i need to change the code to Memory Mapped I/O.
Can you suggest me how to solve this problem?I am using WindowsXP/Vista/7 Os
|
|
|
|
|
|
|
~-!-_= Message =_-!-~
I was born intelligent
Education ruined me!.
|
|
|
|
|
|
|
Hahahaa.... "the Internets..." - George W. Bush (the younger one)
|
|
|
|
|
|
|
Hello,
I find your articles extremely interesting. Maybe I'll write my own driver this month just for fun...
I really appreciate your articles!
I also got the blogging virus..[^]
|
|
|
|
|
|
|
Thank you for your debugging and driver series.
They are quite useful for me.
I think you are a great man and doing an excellent job.
Thank you very much.
dikiili
|
|
|
|
|
|
|
Hi there.
First of all i must say sorry about my bad english, but i'll try to mannage
I'm reading your article 4 the drivers.
I have a task to make a driver for a laser and i have problems.
First problem is that this is new 4 me and i've never wrote a driver before.
How do i prepare a picture 4 the driver to understand?
The laser should be able to burn a picture on wood or metal or stone, but how do i prepare the pictures information 4 the driver? How does the procedure go?
This I think is enough 4 the beginning
I'm new at this and may be commpletly ilogic, but i need all the help i can get
I hope that u can help me with this problem.
Looking forward to your reaplay.
Boban
|
|
|
|
|
|
|
Printer drivers are simmilar to display drivers. Most printer drivers these days run in user mode and call Eng* APIs (Which allows the driver to actually run in user mode or kernel mode since both sets of those APIs are available). You should visit MSDN:
Printer Drivers[^]
8bc7c0ec02c0e404c0cc0680f7018827ebee
|
|
|
|
|
|
|
Toby Opferman wrote:
Leave me a message.
Hello Sir,
your debugging article are cool and intresting but Still i have to graduate to that...
One more compilement Sir, after reading your answer in current thread I am sure you didn't need Debugger to debug your system..
"Opinions are neither right nor wrong. I cannot change your opinion. I can, however, change what influences your opinion." - David Crow
cheers,
Alok Gupta
VC Forum Q&A :- I/ IV
|
|
|
|
|
|
|
Hi Toby,
I am finding you driver tutorial to be very well written and helpful. I have a question about your first tutorial. I was able to get the driver compiled and was able to load and test it withe the other programs. In order to compile the driver I had to replace the macro call TYPE_ALIGNMENT(char) with the constant 1. If I left the macro in I was getting the following errors:
error C4116: unnamed type definition in parentheses
I built the driver in the XP Free environment and I used the bld and the default makefile along with a simple Sources file.
Thanks for your help,
Michael
|
|
|
|
|
|
|
Please contact me. I finally got the budget approved and now can't locate your contact info 
Tom Archer (blog)
Program Manager
MSDN Online (Windows Vista and Visual C++)
MICROSOFT
|
|
|
|
|
|
|
Hi Toby,
I downloaded your code from Driver Development Part 2 and am trying to build. Everything looks okay until the Rebase step. Here's what I'm seeing as output...
rebase.exe -b 0x00400000 -x ..\..\..\..\bin\SYMBOLS -s ..\..\..\..\bin\example
REBASE: *** RelocateImage failed (example). Image may be corrupted
REBASE: Total Size of mapping 0x0000000000000000
REBASE: Range 0x0000000000400000 -0x0000000000400000
NMAKE : fatal error U1077: 'rebase.exe' : return code '0x63'
Stop.
Any idea what's wrong?
Thanks...
Amy 
|
|
|
|
|
|
|
Hi,
Find the line :
$(REBASE) -b 0x00400000 -x $(TARGETDIR) -a $(TARGETDIR)\$(TARGET)
and add ".sys" at the end. So we have :
$(REBASE) -b 0x00400000 -x $(TARGETDIR) -a $(TARGETDIR)\$(TARGET).sys
and everything is alright 
Cheers,
Phil.
|
|
|
|
|
|
|
Hi,sir
So excited to talk to you.
Can I ask you some questions?
Since I want to replace the rdpdd.dll which worked well in windows terminal service, could you tell me how can I do and what will I?
I know the display device of one window is determined by the "default Desktop" of the windowstation. Can I change the Desktop's display device?
Or how Can I create a Desktop with special display device just like rdpdd.dll?
Best regards
fang
|
|
|
|
|
|
|
Hi,
I met another problem:it cant work after installed.
Debugger shows (in appmonitor) hFile==NULL:
hFile = CreateFile("c:\\video.dat", GENERIC_READ | GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, 0, NULL);
Could you tell me how to let it work?
Best regards
fang
|
|
|
|
|
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Rant 
Admin