|
i'd still need to write my own interpreter.
Hey, you'll understand this because you've used TinyPG.
You know how tinyPG lets you put code in the grammars?
That's what this is about.
TinyPG does not do runtime parsing, only generated parsers.
Mine does both. For mine to also do the code in the grammars, and do it at runtime, i'd need to solve this problem.
The only solution i can think of is one that relies on an elaborate caching mechanism for these compiled code bits.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I see....
Gonna think on that....
But what do you have against Emit?
It's no big deal to load of those generated code, is it now? Why does that concerns you?
|
|
|
|
|
I don't have anything against Emit. In fact, I love it.
What I do not love is win32's inability to unload a DLL once it has been mapped to a process' address space.
It's an understandable limitation but one that causes problems if you just start generating code willy nilly under .NET.
What you need to do is ensure you basically never generate the same code twice.
That requires caching. That complicates the code considerably.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I see....
I can understand that could be a problem for very long running processes...
How about running them as Scheduled Task instead of non stop services? Just an idea...
|
|
|
|
|
That's the thing. I don't know how long the process will run. It's an API. It might very well be used in a service.
I much prefer to keep code out of grammars for precisely this problem, but almost all other parser generators support it.
And in using mine, there would be areas where it could come in handy, but if I'm doing it at all, I'm not doing it half cocked. I HATE leaks. Hate them. I come from a C++ world. Leaks are leaks. I don't care that they're not due to a pointer being unfreed - leaking process address memory by constantly loading and throwing away DLLs is a show stopper for me, frankly, in all but the shortest running apps.
So I don't understand why Microsoft is so cavalier about it in .NET. It's nonsense. I'm caching or I'm not doing it at all.
If I'm at least not generating the same code over and over again it's acceptable, but now I'd have to cache the grammars once they're loaded.
On the other hand, I *did* already implement value semantics for all my grammar objects (meaning my grammars themselves can already be dictionary keys)
So I guess it wouldn't be that hard. My problem is thread safety. It's true my system wasn't designed for it, but also I wasn't sharing a process wide cache system either! I at least want the thing to *work* from different threads without causing problems.
 so that's work.
You see what I'm getting at? This to me is bare minimum for what I'd consider acceptable in this scenario.
I assumed Expressions would already take care of that, but if it doesn't even do that, well, I may as well do it all myself and get full code behind support, not just expressions. ah well, it is what it is.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
What if it's all interpreted?
I.e. the piece of code in the grammar is either, generated .cs file or interpreted in memory, i.e. NO EMIT?
This way, no leaks! 
Though it's a bit more work for ya...
|
|
|
|
|
That's why the OP was a rant. I thought the DLR already did that. The truth is, even if I wanted to put in the work, i wouldn't trust C# to be fast enough to do it well where i need it done, and marshalling would kill any performance gain i'd get - that's why i was hoping the DLR already did this..
Like i said, with caching, this is acceptable - i'll take the hit on that as long as when i generate the parsers, the parsers aren't *then* emitting more code. I think i can manage that.
It's just more work and more clunky than i was hoping for. I thought DeclareDynamicMethod/DynamicAssembly was more special than it apparently is
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Adding, I do use reflection in this, but only in one area.
It allows you to define new transform classes to translate grammars to and from different formats.
It uses reflection to enumerate the available transforms and to invoke them.
But that's not time critical code.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
I'm sorry, but reading a paragraph per sentence makes my head hurt trying to string together your thought process here. Are you asking if the DLR supports .NET interpretation? If you are, then yes it does (sort of, as there is still a .NET generation phase going on underneath), as you can see through the DLR supporting REPL - without this capability, Microsoft would not have been able to create IronPython. If you really want to get your head around how it all works, have a read of this[^].
|
|
|
|
|
Can't do compilation here. Because I'm not going to implement the necessary caching to make it realistic.
Just dynamically compiling willy nilly throws a bunch of tiny DLLs into the windows process space and they never get unloaded.
So in order to work around that, one must cache any code they're compiling so they don't recompile the same code twice. Because if they do, their win32 space gets completely polluted with loaded DLLs that are no longer being used. And that number just keeps growing because win32 does not unload DLLs from a process's address space.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
It is possible to have multiple app domains handle this so you could potentially spin up a new AppDomain, run through the interpret phase and execute it, then tear down the AppDomain.
|
|
|
|
|
It's not realistic to do out of process calls during the traversal of a parse table. I count the times between calls in that code in usecs =(
Edit: It might be possible to do it in proc, in a separate AD but that still requires marshalling which makes it heavy.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Unless this is just a rant, you might want to take it to .NET Framework Discussion Boards
"It is easy to decipher extraterrestrial signals after deciphering Javascript and VB6 themselves.", ISanti[ ^]
|
|
|
|
|
it's just a rant. I felt like MS has hyped the DLR unfairly.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Have you had a look at Expression Trees?
|
|
|
|
|
I've seen them. If they work uncompiled i'd be surprised, but even if they do they aren't complete enough to help me implement syntactic predicates in grammars. I'd need statements too.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
honey the monster, codewitch wrote: If they work uncompiled i'd be surprised
They don't. But they can be compiled (and cached) on demand (read runtime).
The rest I didn't understand, but I haven't followed your series of posts because vacation.
What exactly does your statement need to be able to do?
|
|
|
|
|
Like in Coco/R, the parser needs be to able to give control up to a user defined function which has access to a a lookahead buffer of tokens, and returns a boolean value.
The job of the function is to traverse that lookahead buffer looking for patterns in it that tell it how to resolve a grammar conflict.
It's called a syntactic predicate, and it will at least need a while loop of some sort (or a for loop or any kind of iteration statement) and those aren't an expression AFAIK. I mean, I guess i could sort of set up a thing where it called your SP in a loop but i still don't see how that would help.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Expression Trees have iterations and code blocks since .Net 4.
So, yes. I believe it can be done.
I've done similar stuff. (For performance reasons though)
But I have to admit that the more I work with Expressions, the more I appreciate the work the compiler do for us. It's really tedious work.
|
|
|
|
|
I'll look into it but i don't know if i can limit the kinds of queries i need to support to those that can be represented with only expression trees. Although I would certainly prefer that.
But if i did that I'd just go completely declarative anyway
so my grammar construct would look something like
production= identifer [ "<" attributes ">" ] "=" expressions ";" => myexpression
although the => above is NOT a lambda expression. But it's a syntactic predicate in a grammar. I'd probably just use it since it makes sense in this context, but maybe ?? would make more sense. I don't know. Something like that though. =)
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
In case you decide to go down this road, you need to implement some kind of caching functionality, since expressions aren't handled by the GC.
|
|
|
|
|
Ouch, thanks for the warning. I think I'll skip it since the whole point for me was obviating the need for caching.
If i'm going to do caching, I'll just support full code in the predicates. You know?
Gosh that's disappointing, but I'm really glad you warned me. Thanks again.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
|
I will definitely be looking into this.
When I was growin' up, I was the smartest kid I knew. Maybe that was just because I didn't know that many kids. All I know is now I feel the opposite.
|
|
|
|
|
Good luck.
And wear a wig so you don't tear out your own hair.  At least that's how I felt when I started with Expressions.
There are some examples on how to use Expressions in my articles, although the purpose was quite different.
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
