tag:blogger.com,1999:blog-114737972024-02-08T03:14:10.103-08:00Design by ContractHow to design and build a DBC system using C#, and NVelocity.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-11473797.post-1123211862588017992005-08-04T20:13:00.000-07:002005-08-04T20:21:27.790-07:00DBC in useI am finally getting around to making use of Aabs.Dbc in my new open source porject Aabs.Norm. I figured that the best proving ground for a thing like this is a thing like that. That is, a highly complex, high performance application framework, that maintains a state in a variety of very complex ways, that uses polymorphism and .NET in sophisticated ways, and that will be adversely affected if the performance of Aabs.Dbc is poor at runtime.
Here's an example of the interface to the core object - the Persistence Broker.
<style type="text/css">
.csharpcode
{
font-size: xx-small;
color: black;
font-family: Courier New , Courier, Monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }
</style>
<pre class="csharpcode">
<span class="kwrd">namespace</span> Aabs.Norm.Core
{
<span class="kwrd">public</span> <span class="kwrd">interface</span> IPersistenceBroker
{
[Requires(<span class="str">"criteria != null"</span>)]
PersistentObject RetrieveObject(Criteria criteria);
[Requires(<span class="str">"persistentObject != null"</span>)]
[Ensures(<span class="str">"$before(persistentObject) ==
$after(persistentObject)"</span>)]
[Ensures(<span class="str">"persistentObject.IsPersistent == true"</span>)]
[Ensures(<span class="str">"(persistentObject.TimeStamp - DateTime.Now) <
new TimeSpan($time_ms)"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count == 0"</span>)]
IList SaveObject(PersistentObject persistentObject);
[Requires(<span class="str">"persistentObject != null"</span>)]
[Ensures(<span class="str">"$before(persistentObject) ==
$after(persistentObject)"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count == 0"</span>)]
[Ensures(<span class="str">"persistentObject.IsPersistent == false"</span>)]
IList DeleteObject(PersistentObject persistentObject);
[Requires(<span class="str">"criteria != null"</span>)]
[Requires(<span class="str">"criteria.ForClass != null"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList ProcessCriteria(Criteria criteria);
[Requires(<span class="str">"persistentObject != null"</span>)]
[Ensures(<span class="str">"$before(persistentObject) ==
$after(persistentObject)"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Name != persistentObject.
GetType().Name"</span>)]
ClassMap GetClassMapFor(PersistentObject persistentObject);
[Requires(<span class="str">"tableMap != null"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList GetReferrersTo(TableMap tableMap);
[Requires(<span class="str">"persistentObject != null"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList GetLinkedClassMaps(PersistentObject persistentObject);
[Requires(<span class="str">"classMap != null"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList GetLinkedClassMaps(ClassMap classMap);
[Requires(<span class="str">"classMap != null"</span>)]
[Requires(<span class="str">"propertyName != null"</span>)]
[Requires(<span class="str">"propertyName.Length > 0"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList IncomingConnectionsToAttribute(ClassMap classMap,
<span class="kwrd">string</span> propertyName);
[Requires(<span class="str">"procedureName != null"</span>)]
[Requires(<span class="str">"procedureName.Length > 0"</span>)]
[Requires(<span class="str">"type != null"</span>)]
[Requires(<span class="str">"databaseName != null"</span>)]
[Requires(<span class="str">"databaseName.Length > 0"</span>)]
[Requires(<span class="str">"parameters != null"</span>)]
[Requires(<span class="str">"parameters.Count >= 0"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
[Ensures(<span class="str">"$result.Count >= 0"</span>)]
IList ProcessStoredProcedure(<span class="kwrd">string</span> procedureName,
Type type, <span class="kwrd">string</span> databaseName,
NameValueCollection parameters);
[Requires(<span class="str">"procedureName != null"</span>)]
[Requires(<span class="str">"procedureName.Length > 0"</span>)]
[Requires(<span class="str">"type != null"</span>)]
[Requires(<span class="str">"databaseName != null"</span>)]
[Requires(<span class="str">"databaseName.Length > 0"</span>)]
[Requires(<span class="str">"parameters != null"</span>)]
[Requires(<span class="str">"parameters.Count >= 0"</span>)]
[Ensures(<span class="str">"$result != null"</span>)]
<span class="kwrd">string</span> ProcessStoredProcedureRaw(<span class="kwrd">string</span> procedureName,
Type type, <span class="kwrd">string</span> databaseName,
NameValueCollection parameters);
<span class="kwrd">bool</span> ProcessTransaction();
}
}</pre>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com1tag:blogger.com,1999:blog-11473797.post-1120794646666151462005-07-07T20:44:00.000-07:002005-07-07T20:50:46.670-07:00The big day has arrivedWell, it's all out there in the public domain now. Yayyy!!!!!
I've created a project up on sourceforge, and advertised there for help with documentation, installers and general app developers. I've uploaded an initial source code release - and configured my paypal settings.
I can't wait to see how it is received by the public. Pay the site a visit, and let me know what YOU think. And, of course, if you want to make a donation... ;o)
Here's the <a href="https://sourceforge.net/projects/aabsdbc/">SourceForge</a> link.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1120778288371541012005-07-07T16:12:00.000-07:002005-07-07T16:18:08.386-07:00Here goes...I have taken this stuff to the point where it is not too scruffy to release to the world. What I need to do now is publicise it to the world at large. Any ideas about the best forums for that?
I think SourceForgeand gotdotnet will help, but I can also imagine that sharptools and USENET might be good avenues to publicise it through. How will I manage the project when I open the code up to the open source community?
The SourceForge project location is <a href="https://sourceforge.net/projects/aabsdbc/">here</a>.
The GotDotNet project location is <a href="http://www.gotdotnet.com/workspaces/workspace.aspx?id=24fe7e5d-e342-4ad9-a749-8048020f2828">here</a>.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1118569959972334612005-06-12T02:13:00.000-07:002005-06-12T02:52:39.980-07:00Almost ready to go open sourceI have been tweeking and fiddling with the framework in readiness for making it open source. There are still quite a few things I ought to do before making any attempt to publicise it. I thought that since I've been very remiss in not posting to this blog lately I could do some posting. How better than to list what I've got to do before the system is unveiled to the public? I know I'm not exactly going about this in the true open-source way, but I guess I'm a little wary about releasing my code to the public without having crossed as many 't's and dotted as many 'i's as I can find. I'm sure there are plenty of improvements that people will find when I release them, but I don't want any of them to be dumb or outmoded errors!
Anyway, enough of such insecurities! Here's what I have to do before D-day:
<ul><li>See if I can create a WIX installer that works. Currently I seem to be having problems with adding assemblies to the GAC. My best guess is that WIX is not able to add .NET 2.0 assemblies to the GAC because it's trying to use some sort of reflection on an incompatible assembly.</li><li>Create a user tutorial. I'll probably do that in this blog. It should be a pretty easy task, since I've gone to great lengths to make the framework as transparent as possible.</li><li>Document the API using NDoc. As ever, I've been slack in commenting my code. I guess I'll start with the main APIs and then keep at it in the months after release.</li><li>Run it through FxCop and make sure there are no obvious errors.</li><li>Run regression tests on the static code generation system! This is potentially of less value than the dynamic code generation system so I have ignored it for a while.</li><li>Make sure it still compiles on .NET 1.1! And how about Mono? I haven't seen any major commercial projects requiring Mono, but that may change...</li><li>Make sure that the compilation with NAnt works as well as the VS.NET 2005 beta 2 builds. Especially if WIX is not an option. I need to make tragets that will crerate binary and source distributions for release. This should then be used to upload overnight snapshots to the web each night.</li><li>Convert the system to work with CruiseControl.NET. I have (so far) had around 5 interviews with ThoughtWorks, so I really ought to be as obsequious as possible to boost my chances with them! ;-)</li></ul><p>Once these tasks have been performed, then I ought to post it online and start trying to publicise it a bit more.
I can then think a bit more about using the framework in earnest. I want this system because I have an object relation mapping (ORM) system that is just short of being releasable and I would like to augment it with DBC to see whether I can productise it that way. For more information on ORMs, google for Scott Ambler's original postings on a design for a robust persistence layer. I also want to make the second release of the framework self-hosting. That is - I want to use DBC within the DBC framework itself. How much more could I practice what I preach, eh?</p>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1118564486388483862005-06-12T01:06:00.000-07:002005-06-12T02:14:36.886-07:00May was a quiet monthIt's been almost a month since the last time I posted, and I have made a few changes to the DBC framework since then. I'm making changes now in readiness for release to GotDotNet or Sourceforge. So I'm just making sure everything is as logical and consistent as I can make it.
Firstly, I have completely rewritten the NVelocity template assembly to simplify it even more (as if that were possible!). Now you can use it like this:
<pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">void</span> TestDefaultCreationOfTemplate()
{
ITemplate tpl = TemplateManager.CreateTemplate(<span class="str">@"Hello" +
</span><span class="str">" ${otherpart}"</span>);
tpl[<span class="str">"otherpart"</span>] = <span class="str">"world"</span>;
Assert.AreEqual(tpl.ProcessedOutput, <span class="str">"Hello world"</span>,
<span class="str">"strings didn't match"</span>);
}</pre>The observant amongst you will also notice that I'm now using unit test cases from visual studio 2005's unit testing framework - why MS couldn't just use NUnit I don't know (well I do, and it doesn't reflect well on them), especially since their framework seems excessively complicated when using autogenerated tests.
Anyway, the template class is now just a very thin veneer over the velocity framework. I am still able, should the need arise, to swap out NVelocity and replace it with something else, but the changeover was mostly involved in recompilation of NVelocity to .NET 2.0 and to add a strong name for deployment to the GAC.
I've placed the code generation templates into the Core assembly as resources to allow easier deployment and to make them tamper proof and to get rid of the needless configuration settings to allow the system to find the templates.
I've broken out the predicate attributes and the exception types to a separate assembly, so development can be done with DBC attrs even if the DBC system is not in use. It also means the attributes can be put to other uses like documentation or validation.
Right now, I'm mostly in performance tweeking and refactoring mode. Here's an example of one of the tests I'm using that shows how the system is to be used now:
<pre></pre><pre class="csharpcode">[TestMethod]
<span class="kwrd">public</span> <span class="kwrd">void</span> TestTimeToCreateAThousandProxyObjectPairs()
{
ArrayList al = <span class="kwrd">new</span> ArrayList();
DbcProxyDynRTCF cf = <span class="kwrd">new</span> DbcProxyDynRTCF();
<span class="kwrd">int</span> totalToCreate = 10000;
cf.CreateProxyFor(<span class="kwrd">typeof</span>(TestObjectWithEvents));
DateTime start = DateTime.Now;
<span class="kwrd">for</span> (<span class="kwrd">int</span> i = 0; i < totalToCreate; i++)
{
al.Add(cf.CreateProxyFor(<span class="kwrd">typeof</span>(TestObjectWithEvents)));
}
TimeSpan durn = DateTime.Now - start;
<span class="kwrd">double</span> d = durn.TotalMilliseconds / ((<span class="kwrd">double</span>)totalToCreate);
Debug.WriteLine(<span class="str">"Average time to create a proxy-object pair" +
</span><span class="str">" (ms): "</span> + d.ToString());
Assert.IsTrue(durn.TotalMilliseconds < ((<span class="kwrd">double</span>)
totalToCreate) * .17);
}</pre>Currently the average time needed to instantiate a proxy with a target object is around .2 milliseconds. Interestingly it was around 0.16ms with .NET 1.1, but now I'm working with .NET 2.0 the performance is roughly 25% slower. It may be that some of the techniques I am using can be performed in a better way now, but that remains to be seen -- the only deprecated feature that I have not updated is in my use of CreateCompiler to compile the generated code. This piece of code gets called only once, before the timer is turned on. So it couldn't affect the code performance unduly. So where is this much publicised performance boost that we are supposed to get from .NET 2.0?
<pre class="csharpcode"><span class="kwrd">public</span> CompilerResults GenerateAssembly()
{
Assert(SourceCollection != <span class="kwrd">null</span> && SourceCollection.
Count > 0);
AssertCodeGenParamsAreOK();
CSharpCodeProvider cscp = <span class="kwrd">new</span> CSharpCodeProvider();
ICodeCompiler compiler = cscp.CreateCompiler();
CompilerParameters cp = ConstructCompilerParameters();
<span class="kwrd">string</span>[] classes = <span class="kwrd">new</span> <span class="kwrd">string</span>[SourceCollection.Count];
SourceCollection.CopyTo(classes, 0);
CompilerResults cr = compiler.CompileAssemblyFromSourceBatch
(cp, classes);
<span class="kwrd">if</span> (cr.Errors.HasErrors)
{
<span class="kwrd">foreach</span> (CompilerError ce <span class="kwrd">in</span> cr.Errors)
{
Log(ce.ErrorText);
}
}
<span class="kwrd">return</span> cr;
}
</pre>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1115779794080609502005-05-10T19:44:00.000-07:002005-05-10T19:49:54.126-07:00DBC & AOP don't mixThis time I'm going to explore some of the issues I've found in converting the prototype over to work in a JIT environment. If you've seen any of my previous posts, you'll know that I have a working prototype that uses static code generation to produce proxies to wrap objects that identify themselves with the DbcAttribute. These proxies are produced solely for those objects, so the developer has to know which items have predicates on them in order to instantiate the proxies rather than the target objects. If the programmer forgets, or doesn't know, that an object has predicates, he may create the target object and not benefit from the DbcProxy capabilities.
I've therefore been looking at ways to guarantee that the proxy gets generated if the library designer has used DBC. There are a lot of ways to do this, and having not explored all of them, I want to find a way to construct the proxies that will allow any of these approaches to be employed at run time. That naturally makes me think of abstract class factories and abstraction frameworks. It is nice to allow the library designer to define predicates on interfaces, but that may not fit the design they are working on, so I don't want to mandate it.
The solution I've looking at uses the code injection system that comes with the System.Enterprise namespace's ContextBoundObject classes. This framework guarantees that the object gets a proxy if it has an attribute that derives from ProxyAttributes. I would derive my DbcAttribute from ProxyAttribute and get an automatic request to create a proxy. Sounds perfect? Yes! But as you will see, there is a gotcha that will prevent this approach. Read on to find out more...
<h4>Status of the prototype</h4>
The prototype uses a Windows console application that is invoked as part of the build process to create the proxies for an assembly. The program is invoked in association with an application config file to provide it with the details of the target assembly, where the resulting proxy assembly is to be stored, and what it's to be called. It has a few other config settings to define how it will name the proxy for a target object, and what namespace it shall live in. So far, I have not added any guidance to it on how to handle the assemblies in a not static way. I haven't implemented any dispenser to intercede in the object creation process. In this blog I'm going to take a look at what config settings I need to add to control the choice between dynamic and static proxy generation.
I'm using the Microsoft Enterprise Application Blocks to handle configuration, and I'm using log4net to take care of logging issues. NVelocity is being used to construct the source code for the proxy. None of that will change in the refactoring that is to follow. In fact, very little has to change in the event model to turn the system into a dual static/dynamic proxy generator. What changes is in the issue of how to handle the prodicates when you are using a dispatch interface. More on that to follow.
<h4>static code gen via CSharp compiler</h4> The static code generation process has been described in a quite a bit of detail in previous posts, but in essence it works by scanning through all of the types in an assembly, looking for ones that are adorned wit the DbcAttribute. For each of those, it scans through each of the InvariantAttributes, and then through each of the members in the class to get their details and any predicates that are attached to them. It then creates a proxy class that imitates the target type, except that prior to each invocation it checks a set of pre-conditions, and afterwords checks post-conditions. This proxy type is kept around until all of the assemblies types have been scanned, and then a new proxy assembly is compiled and saved to disk. This Proxy assembly is then used instead of the target type.
The CSharpCompiler provides a set of options to allow you to keep the source code used to create the assembly, and to create debug information to tie the assembly to the source code at run-time. This is useful for debugging the DBC system, but should not be necessary when using the framework in the field. If errors occur they will be on either side of the proxy (assuming I debug the proxies properly) and the user shouldn't want to see the source for the proxy itself. We therefore need to be able to turn this off, along with debug information.
Another feature of the compiler is the ability to dispense an in-memory only assembly that is available for the lifetime of the process. This is just what we need for dynamic proxy generation.
<h4>New library for dynamic proxies</h4> One refactoring that is inevitable is that we now need two front-ends to the DBC system. A command line tool for static proxy generation that can be invoked from VS.NET or NAnt. We also need something that can be used in-proc to dispense dynamic proxies on demand. This can't be another process unless you want to use remoting to access your objects. You may want to do that, but you need to have some control of that situation, and it would be a little intrusive to have that decision taken away from you.
<h4>Dynamic proxy code generation pattern</h4> The process for the dynamic proxy is much the same as for the static proxy, it just tweaks the compiler settings a little, and provides some speed ups for handling the storage and retrieval of assemblies for types that have already been code generated. The procedure is as follows:
<ul>
<li>client requests an objects of type X</li> <li>creation process is intercepted by the CLR, because the DbcAttributes is declared on the object or one of ins ancestors.</li> <li>The DbcAttributes provides an object that can be used to intercept calls to the object, and it passes the creation of that object off to the dynamic proxy code generator.</li> <li>The dynamic proxy CG checks in a HashTable whether the target type has been requested previously, if so it instantiates the generated proxy and returns.</li> <li>if no type has been generated for this type it creates a scanner and a code generator as per the static proxy CG, and requests the scanner to scan the type.</li> <li>After the type has been scanned, there will be exactly one element in the ProcessedTypes context variable in the CG.</li> <li>The dynamic proxy CG then requests that the CG generate an assembly for the type, specifying that the assembly must be in-memory only.</li> <li>The assembly is generated, and it is stored along with the target type in the HashTable for future reference.</li> <li>The new proxy type is dispensed by the DbcAttribute, and the target object creation commences as normal.</li> <li>Next time a call on the target object is performed the proxy object is allowed to intercept the call, and run its checks.</li> <li>voila!</li>
</ul>
This seems almost exactly how I want the dynamic proxy to work. The client doesn't have an option but to use it if it is available. This convenience comes at a price. That price is detachment from the state of the target object. If you want to do any interesting kinds of test on the current state of the target object, you need a reference access to the various public members of the type.
Without access to the target instance you can't make pre or post or invariant checks, because you can't access the state to see how it has changed or take snapshots of the initial state. It is in effect unusable. So does that mean that the dynamic proxy is incompatible with DBC?
The problem here is not in the idea of a dynamically generated proxy, it is in the <i>kind</i> of proxy we are using, and in its relationship to the target object. It is not connected to the target object in any way - it could even be on a different machine from the target. Even if it was directly connected (within the enabling framework) it wouldn't be of much use since it is designed to deal in IMessage objects, not target object instances.
The IMessage instance is a property bag that contains name value pairs for each of the parameters that is passed into a single method called "Invoke" which is able to then process the parameters. This pattern will be familiar to those who ever wrote COM components that needed to be compatible with VB - it is a dispatch interface. This sort of thing is great if you want to log the values of parameter calls, or to perform rudimentary checks on the incoming parameters.
We want to go a lot further than that. You can't make many assertions about how an object behaves just by describing its input and output values. And <b>that's the problem that I have with standard interfaces</b> - apart from a name which may or may not mean anything they only have a signature. A signature is just a list of the parameters and return types. Nice to have obviously, but not a lot of value when it comes time to work out what happens inside the implementation(s) of the signature.
So, we need direct access to the target object to be able to go any further. Does that mean that the context bound object approach is dead in the water. I'm not sure and this is fairly sparsely documented area of the framework so I haven't been able to track down an answer to that question yet. Answers on a postcard if you know a definitive answer to this. I am going to assume so for the time being. This hasn't been a wasted time - we've learned a useful fact about the limitations of dynamic proxies - namely that they are not compatible with aspect oriented programming paradigms. If anything they can the end-point of an AOP proxy chain, but they cannot sit anywhere in the middle of a chain of responsibility. It's a depressing fact of life - but useful to know.
What else can we deduce from this? We know that any kind of DBC proxy is going to need access to the state that is somehow deterministic. The key concept is state here. Anything that transforms the state of the target object also changes the semantics. Imagine, for example that the DBC proxy sat at the far end of an AOP chain - it will have an instance to a target object but the target object will delegate all the way down to the real target object at the other end of the chain. All of the state inquiry methods are present, but are converted into IMessage objects and passed through the chain of proxies before arriving at the real target object. The AOP chain can modify the data going into the real target, and modify the results coming out of it. They can also change the timing characteristic making a synchronous call where previously the interface was not, or vice versa. They could transform XML documents or add and subtract values from parameters. The point is, when you are out of touch with the target object, you have no guarantees event hat the parameters you are making tests against are the same as are received by the real target object. You consequently face a situation where the semantics are similar but potentially subtly modified, and you have to track that in your assertions.
This may sound like a rather academic discussion of the potential to modify an interface to have diferent semantics, but that is the essence of the adapter design pattern, and is an occupational hazard with SOA architectures. Don't forget that when you invoke an API on a third party software system, you think you know what is happening, but you often don't. You generally have some documentation that acts as a contract saying what goes on inside the service when the service gets invoked. We all know that such documentation often gets out of date, and we also know that there is little you can do to stop a well-meaning-fool from breaking your system. That's Murphy's law at work again. If something can go wrong, it generally will. And normally it is introduced by someone who thought they were solving some other problem at the time.
So published interfaces aren't worth the paper they are written on, especially if you're playing a game of chinese wispers through layers of other people's code. We need a system that is direct and compulsory, with no middlemen to dilute the strength of the assertions that you make on a component.
Next time I will describe how I solve this problem - probably by paying homage to the <a href="http://en.wikipedia.org/wiki/Design_Patterns">Gang Of Four</a>...aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1114560213102612902005-04-26T16:24:00.000-07:002005-04-26T17:15:54.116-07:00Predicates and Expressions - Part 2Last time I started discussing the expression language used to define predicates on types and members. My conclusions were:
<ul>
<li>C# expression syntax used since code must be transformed to that in the end, and the more similar the syntaxes of the expression language - the easier the transformation will be.
<li>No side-effects. Side effect within the predicates complicate the tasks of determining whether a temporal comparison is actually effective. If the snapshots can't be guaranteed to be faithful representations of the state before and after the test, then you can't trust the tests, and you might as well not have bothered with them in the first place.
<li>Pragmas, or pre-processor messages, are used in expressions to add a temporal ability to the predicates. <span style="font-family:courier new;">$before(foo)</span> allows a snapshot of the variable <span style="font-family:courier new;">foo</span> to be taken for later comparison with the exit state.
<li>Pragmas may map to identity in some circumstances. Entry states can have no <span style="font-family:courier new;">$after()</span> since we haven't reached the exit state yet. Likewise, there is no notion <span style="font-family:courier new;">$before()</span> state since this is the before state, and we can't take a snapshot of the state before the before state (unless it's on some other superior method). Therefore, in a precondition it should be illegal to insert pragmas into predicate expressions. In some cases such as <span style="font-family:courier new;">$before(foo)</span> in a precondition, and <span style="font-family:courier new;">$after(bar)</span> in a post condition the pragmas map onto the current value of the variable. i.e. in a post condition <span style="font-family:courier new;">$after(bar)</span> can be transformed into <span style="font-family:courier new;">bar</span>.
</li></ul>
The algorithm for converting predicates to guard code is shown below.
<pre><span style="font-size:78%;">
foreach assembly in assembly list
foreach type in assembly
clear list of invariance predicates
clear list of processed methods
foreach invariant attribute on type
transform pragmas on predicate
add pre/post analysis objs to list
generate code for predicate
append code to processedInvariants list
end foreach invariant
foreach member in type
clear list of requires attrs
foreach requires attr on member
transform pragmas on predicate
add pre/post analysis objs to list
generate code for predicate
append code to processedRequires list
end foreach requires attr
clear list of ensures attrs
foreach ensures attr on member
transform pragmas on predicate
extract list of vars to be snapshot
add to list processedSnapshots
add pre/post analysis objs to list
generate code for predicate
append code to processedEnsures list
end foreach ensures attr
insert snapshots into code template
insert pre/post analysis objs to template
insert code for requires guards into template
insert code for ensures guards into template
generate code
add generated code to processed methods list
end foreach member
insert code for members into type template
clear list of invariance predicates
generate code for type
append code to processed types list
end foreach type
setup c# compiler with config params
compile all types in processed types list
store assembly as per instructions
end foreach assembly </span>
</pre>
This algorithm conveniently glosses over the complexities of the depth first search event propagation mechanism described in previous posts, but the outcome is the same ultimately.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1114389678145135932005-04-24T17:28:00.000-07:002005-04-26T17:06:08.796-07:00Predicates and Expressions - Part 1The predicates that are used to specify the <em>in</em>, <em>out</em> and <em>inv</em> conditions on a component need to follow certain rules. The first and least obvious is that it must follow the syntax and semantic rules of the C# language. I say that it is the least obvious, since it seems to be conventional to invent a new predicate language for these purposes. I can see the point of this, since it would allow a tailor made language to be used, and to extend it beyond the capabilities of a normal C# expression. I intend to start off with the syntax of C# because they will have to end up as C# expressions, and I want to have to intervene in the simplest way possible. I want the intervention to be achievable via regex replacements where possible.
It it essential to make one significant enhancement to the expression language of C#. It needs to be temporal. It needs to be able to make statements about the value of a variable before and after a method has been invoked. In a temporal logic that might be represented by a prime (i.e. <span style="font-family:courier new;">x'</span>). That could get rather confusing when we are making comparisons with characters that need to be enclosed by single quotes. At the very least it would be hard to read. Likewise we could represent it as a function which can be made available through a supertype or a utility object. An expression might resemble this: "<span style="font-family:courier new;">before(x) == after(x) - 1</span>". The problem is that this pollutes the method namespace so we would have to have something like this in the expressions: "<span style="font-family:courier new;">base.before(x)</span>". Which is already looking ugly, and it also forces the user to inherit from a supertype that may wreck their application design.
It seems that we need <i>pragmas</i> - preprocessor directives that can be used as clues to the framework. Since we are using the syntax of C# we can be sure that neither types nor member names will begin with a dollar sign. It seems safe to assume that we can begin pragmas with a dollar sign like so: "<span style="font-family:courier new;">$before(x) == $after(x) + 1</span>". It should not be confused with interpreted scripting language though, this is input for a pre-processor that outputs code, which is subsequently compiled. That applies whether we are performing dynamic code generation.
The sharp-eyed of you will notice that in various kinds of predicate the <span style="font-family:courier new;">$before(x)</span> and <span style="font-family:courier new;">$after(x)</span> are synonymous with <strong><em><span style="font-family:courier new;">x</span></em></strong> itself. For example it makes no sense to refer to a <span style="font-family:courier new;">$after(x)</span> in a <i>requires</i> predicate. Likewise, there isn't much use for an <span style="font-family:courier new;">$after()</span> pragma since we are inserting code into the method of a target object. <span style="font-family:courier new;">x</span> will already embody the value in <span style="font-family:courier new;">$after(x)</span> so the <span style="font-family:courier new;">$after()</span> will just be getting in the way. So, we only need a <span style="font-family:courier new;">$before(x)</span> pragma
to take a snapshot of the value of the variable at the beginning of the method, for later comparison with what the variable becomes.
There are some rules that predicates must obey, and we just saw one of them before. In a pre-condition,<span style="font-family:courier new;"> $before(x)</span> and <span style="font-family:courier new;">x</span> are synonymous, and therefore <span style="font-family:courier new;">$before()</span> is meaningless, and shouldn't be found there. This sort of rule can be detected and converted into harmless code by the preprocessor. Another hard and fast rule is that predicates must <b><em>never</em></b> have side-effects. That is - the state before testing the predicate should be exactly the same as the state after the test. There are circumstances in which that is untrue. If a predicate detects a bug, it may throw an exception, which may cause the stack to unwind, which will have a side effect. But that is a desired side effect that is part of the semantics of design by contract and error detection, as well as being permissible in a constant method (assuming such a thing were possible in C#). What I mean is <i>undesired side-effects</i> - side effects that are not implicit in the error checking framework, and which are not testable by the predicates themselves.
There is a need to avoid harmful code that is harder to spot. The predicates should be without side-effects. It's easy enough to spot when someone accidentally inserts a "=" instead of a "==", but if they code a helper method such as "<span style="font-family:courier new;">IsOkToDoIt(this)</span>" that happens to change a variable as it runs, then the predicates change from being an external test of correctness into another kind of source code. We need to prevent this - we are interested in making statements about the state of the class, and how it changes from moment to moment. We don't need to be concerned with making changes to it, since that is what the component code is for.
There is no way to prevent the predicates from having a side effect. Fullstop! Surprisingly C# doesn't support the ability to define constant methods. Constant methods are a powerful feature in the armoury of the C++ library writer. It makes it possible to assert the fact that a method has no side effects. In C# you are not able to make that kind of assertion. So we will just have to make it known to the DBC-driven programmer that coding predicates that have side-effects is asking for misery.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com1tag:blogger.com,1999:blog-11473797.post-1113991869134641482005-04-20T03:04:00.000-07:002005-04-20T03:11:09.136-07:00An interesting propositionAs you will recall from the previous post, I have been wondering about how to implement the functionality for dynamic proxies. Well I saw <a href="http://www.theserverside.net/articles/showarticle.tss?id=AspectOrientingNET">this </a>great article, by Viji Sarathy, on the web, and think that this might be the way to go.
It uses Context Bound Objects to ensure the interception of all calls by the CLR without any explicit framework setup by the user. We can guarantee that the contracts get followed, whether the user tries to evade them or not.
The only misgiving I have is over the performance characteristics of the thing. Any thoughts?aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1113352034132352672005-04-12T17:00:00.000-07:002005-04-12T17:27:14.143-07:00Code Generation Patterns - Part 3<p>This time I'm going to describe the source snippet that I posted last time. In previous posts I showed the implementation that I use of the Observer pattern to fire off events notifying the code generator about code structures that it has found. The code generator was shown last time, and you can see that there are a whole bunch of event handlers that take note of the code structures and perform code generation tasks as required. </p><p>A typical example is the handler <i>NewType</i> that generated code for classes, structures and interfaces. As you will recall from my last post, I am using depth-first recursion in my scanner, to allow my code generator to generate in one pass. That means that there will be a lot of generated code knocking about from all of the invariant predicates, methods, events, fields and properties that were detected before the type event was invoked.</p><p>The first thing the CG does is add the namespace of the type to a list of Namespaces that is being kept for addition at the top of the code for the proxy. Obviously the proxy needs to know about the target object. As a general rule you should use the fully qualified name of any type that you refere to in argument lists, return types etc. Readability is not an issue here (very much) but not having to keep track of all the namespace references that could be needed is worth this stricture.<p>Following the namespace manipulations the generated code is added to the context variables for the NVelocity template. The generated code is kept in the context vars between lines 10 and 21. For common code structures such as assemblies, types and methods the generated code is added to an ArrayList and when the next most senior CG template needs to include them, it just iterates through the list, inserting everything that it finds. The HashTables are an implementation convenience that helps the CG to maintain a unique list of variables that need to have snapshots taken.</p><p>After the CG has added all of the snippets of code that have been generated so far, it invokes the template for the type itself. The template looks like this:
<pre><span style="font-size:78%;">namespace ${type.Namespace}.Proxies{
#region Class ${type.FullName}Proxy
public class ${type.Name}Proxy
{
internal ${type.FullName} TheTargetObject = null;
public ${type.Name}Proxy(${type.FullName} target)
{
TheTargetObject = target;
}
#foreach($method in $methods)
$method
#end
#foreach($property in $properties)
$property
#end
#foreach($field in $fields)
$field
#end
#foreach($event in $events)
$event
#end
} // end class ${type.FullName}Proxy
} // end namespace ${type.Namespace}.Proxies
#endregion</span>
</pre>
<p>As you can see the template is minimal. All of the interesting work has been done in other templates or "<em>Pragma processors</em>". I'll be going into pragma processors in depth next time. Work ahead of me includes making the proxy implement the same interfaces as the target object. In the template above the proxy class is generated with exactly the same name as the target class, but with "<em>Proxy</em>" appended. It has the same namespace with "<em>.Proxies</em>" appended. The proxy stores a reference to the target object called <em>TheTargetObject</em>. TheTargetObject is delegated to when all the predicate checks have been performed. The template for a method is where the action is:</p>
<pre>
<span style="font-size:78%;">#if(!$utils.IsProperty($method))
#if($utils.IsVoidType($method.ReturnType))
#set($returntype = "void")
// found a void return type
#else
#set($returntype = ${method.ReturnType.FullName})
// didnt find a void return type
#end## if isVoidType
public new $returntype ${method.Name}(#set($comma = "")
#foreach($arg in ${method.GetParameters()})
$comma ${arg.ParameterType.FullName} ${arg.Name}#set($comma = ", ")#end){
// take the 'before' snapshots
#foreach($snapshot in $ssbefore)
$snapshot.TypeName ${snapshot.After} = ${snapshot.Before};
#end
//TODO: Invariant code here
#foreach($inv in $invariants)
$inv
#end
//TODO: Require code here
#foreach($req in $requires)
$req
#end
## now the call the the real object
#if(!$utils.IsVoidType($method.ReturnType))
$returntype result =
#end##if is void
#if($method.IsStatic)
${fullclsname}.${method.Name}(#set($comma = "")
#else
TheTargetObject.${method.Name}(#set($comma = "")
#end
#foreach($arg in ${method.GetParameters()})
$comma${arg.Name}#set($comma = ", ")#end);
// take the 'after' snapshots
#foreach($snapshot in $ssafter)
$snapshot.TypeName ${snapshot.After} = ${snapshot.Before};
#end
//TODO: Ensure code here
#foreach($ens in $ensures)
$ens
#end
//TODO: Invariant code here
#foreach($inv in $invariants)
$inv
#end
#if(!$utils.IsVoidType($method.ReturnType))
return result;
#end##if is void
}#end ## if is not property
</span></pre>
<p>The templates are pre-loaded with a "<em>utils</em>" object that is able to perform various jobs related to analyzing types that are beyond the capabilities of NVelocity. We use this to check whether the method is a property or not. Properties in C# are handled as methods in the CLR. If we ignore properties and generate them as methods we lose a lot of the syntactic flexibility that C# makes available.</p> <p>We need to check then whether the method is void or not. If so, we provide the short form of the void type, since the code will not compile if we use "System.Void". <em>Not sure why that is the case. Answers on a postcode please!</em></p><p>Having determined what the return type is going to be, we can then create the method signature. We are only ever going to get notified about the public methods, since those are all that external users of the proxy or the target object would ever get to see. We don't need to wrap protected, private or internal methods so we can hard-code the public access specifier. Next, the method parameters are inserted. They are a complete copy of the arguments of the target objects, but with the type names expanded.</p><p>Now we get to the whole purpose of this exercise - to insert predicate checks into the method body to check parameters, members and other features before and after invocation of the method on the target object. the pattern of the invariants is as follows:</p><ul><li>invariant 1 </li><li>invariant 2</li><li>invariant ... </li><li>invariant n </li><li>require 1 </li><li>require 2 </li><li>require ... </li><li>require n </li><li>TheTargetObject.Method(...) </li><li>ensure 1 </li><li>ensure 2 </li><li>ensure ... </li><li>ensure n </li><li>invariant 1</li><li>invariant 2 </li><li>invariant ... </li><li>invariant n</li>
</ul><p><em>Invariants </em>are predicates that must be true at all times - they must be true before and after the method has been called. Therefore we bracket the call with the invariant tests, and have <em>requires </em> checks before and the <em>ensure </em>checks after. For more information on these predicates take a look at the language documentation for Eiffel. Since the invariants are invoked before and after every method call you should make sure that they are not too expensive.</p><p>At the end of the method template the script looks again to check whether the method returns a void, and if so skips the returning of a result. It also doesn't bother to collect the result in the first place.</p><p>Each of the methods is built this way, converted into a string and then appended to the end of the <em>ProcessedMethods</em> properties in the CG. When the <em>type event</em> is finally emitted, the ProcessedMethods property contains a wrapper for each of the methods that the scanner found. That should be every public method that the target object implements or inherits. Obviously, the invariant properties of the object must be true for <em>all</em> properties, fields, events and methods, so it is not enough just to wrap the methods that have <em>Require </em>and <em>Ensure </em>attributes attached, since the <em>Invariant </em>attributes apply for the whole type at all times.</p><p>Next time I'll show you how I convert the assertions made in the invariant, require and ensure attributes into working code that can be inserted into the proxy. I'll leave you with this exercise - How would <em>you</em> convert this into a peice of working code. I'll show you how I do it. Let me know if your approach is better, because I'd be glad to use it!</p>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1113034012023631562005-04-09T00:59:00.000-07:002005-04-12T17:35:08.423-07:00Code Generation Patterns - Part 2<p>Last time I described typed and un-typed events, and multicasting events to several listeners. One of those listeners would be a code generator. This time I'll go on to describe some of the necessary infrastructure that such a design requires. Generally this information is all about tieing the events together. The context in which an event takes place is as important as the event itself. </p><p>When you're generating code progressively, you need to keep track of what you have done, and what still needs to be done. In this case that means the code already generated, and the messages already received, but have not generated code for. It also indicates what code structure(s) an incoming predicate applies to. </p><p>There are two complementary modes of operation that a scanner/code-generator system can use for <i>broadcasting</i> events. They are variations of tree search algorithms. In the first case, the most senior element is broadcast first. that means that an event would be generated for a type before the events for its methods are fired. I shall call this (for want of a better name) the "<strong><em>depth-last recursive search</em></strong>". The second approach is a true "<strong><em>depth-first search</em></strong>" since elements that are lowest in the object tree are announced sooner. These two modes support different code-generation plan. The choice will have an effect on what sort of state one has to keep, and how long it has to hang around. More on that later.</p><p>With depth-first recursion a <em>method event</em> will be broadcast before the corresponding <em>type event</em> , and a predicate such as an <i><b>Ensure</b></i> attribute, will be received before the method on which it was attached. Therefore, when you are generating code, you can't just slot what you get into what you already have, because you don't have anywhere to put it. </p><center>
<img src="http://aabs.aspxconnection.com/objectTree.png" />
<b>An object tree. Depth means further from the assembly object.</b>
</center><p></p><p>With a depth first search context variables track the predicates detected till an event for the method is broadcast and we can generate the whole code for the method. We still have to keep the generated code around till we get the type event! Needless to say, we could hunt for such information as the method event arrives. We could use reflection to navigate up the type object graph as far as the assembly if we wanted to. But if we rely too much on that sort of navigation we break the whole broadcast idiom, and morph the algorithm into a depth-last recursive search (which has it's own unique requirements that I'll come onto next).</p><p>In the depth-last search the event for the more senior object is fired before that of its subordinates. That means we get the event for the type before that of the method, and the event for the method before that of its predicates. That's helpful because we now have something to hang the subsequent events off of. If you were producing a whole object graph in memory then this would be ideal, since the tree would always be properly connected rather than fractured. This approach is not without its drawbacks, not least of which is that you have to build up an object graph in memory before you can generate the code for it! With depth-first recursion you know that when you get the event for the method that there are no more predicates coming. You know when it's safe to generate code. With the depth last approach you have to send a "<i>finish</i>" message that says when the stream of sub-objects has stopped. On the whole I've found for this project that depth-first navigation of the assembly object graph works fine, and simplifies the event interface of the listeners that I want to use. In other projects I've used this on I've done the opposite, and everything has gone OK, it really depends on the sizes of the data stream, and the performance characteristics required. There are drawbacks with either approach and you should probably decide on the basis of the following criteria:</p><ul><li>Is the stream of events large (or never-ending) <li>Do you need to keep the resultant data structures around in memory <li>Would single-pass processing be an advantage to you.</li></ul><p>The snippet below shows some of the code that I use to process events in the code generator. Much has been chopped out of course, but from this you should see how all the talk about depth-first searches and events translates into code generation.
</p>
<!-- code formatted by http://manoli.net/csharpformat/ --><pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">class</span> DbcProxyCodeGenerator : DbcSupertype
{
<span class="kwrd">public</span> DbcProxyCodeGenerator()
{
InitialiseTemplates();
}
<span class="preproc">#region</span> Context Variables
<span class="kwrd">private</span> ArrayList processedAssembly = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedTypes = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedMethods = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedProperties = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedFields = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedEvents = <span class="kwrd">null</span>;
<span class="kwrd">private</span> Hashtable processedSnapshotsBefore = <span class="kwrd">null</span>;
<span class="kwrd">private</span> Hashtable processedSnapshotsAfter = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedInvariants = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedEnsures = <span class="kwrd">null</span>;
<span class="kwrd">private</span> ArrayList processedRequires = <span class="kwrd">null</span>;
<span class="kwrd">private</span> Hashtable processedNamespaces;
<span class="kwrd">public</span> <span class="kwrd">void</span> NewAssembly(<span class="kwrd">object</span> sender,
NewAssemblyEventArgs e)
{
vtAssembly.SetAttr(<span class="str">"assembly"</span>, e.TheAssembly);
<span class="kwrd">string</span>[] namespaces = <span class="kwrd">new</span> <span class="kwrd">string</span>[
ProcessedNamespaces.Keys.Count];
ProcessedNamespaces.Keys.CopyTo(namespaces, 0);
vtAssembly.SetAttr(<span class="str">"namespaces"</span>, namespaces);
vtAssembly.SetAttr(<span class="str">"types"</span>, ProcessedTypes);
ProcessedAssembly.Add(vtAssembly.Merge());
}
<span class="kwrd">public</span> <span class="kwrd">void</span> NewType(<span class="kwrd">object</span> sender, NewTypeEventArgs e)
{
ProcessedNamespaces.Add(e.TheType.Namespace, <span class="kwrd">null</span>);
vtType.SetAttr(<span class="str">"type"</span>, e.TheType);
vtType.SetAttr(<span class="str">"methods"</span>, ProcessedMethods);
vtType.SetAttr(<span class="str">"fields"</span>, ProcessedFields);
vtType.SetAttr(<span class="str">"properties"</span>, ProcessedProperties);
vtType.SetAttr(<span class="str">"events"</span>, ProcessedEvents);
ProcessedTypes.Add(vtType.Merge());
ProcessedMethods = <span class="kwrd">null</span>;
ProcessedFields = <span class="kwrd">null</span>;
ProcessedProperties = <span class="kwrd">null</span>;
ProcessedEvents = <span class="kwrd">null</span>;
ProcessedInvariants = <span class="kwrd">null</span>;
}
<span class="kwrd">public</span> <span class="kwrd">void</span> NewMethod(<span class="kwrd">object</span> sender, NewMethodEventArgs e)
{
vtMethod.SetAttr(<span class="str">"method"</span>, e.Method);
vtMethod.SetAttr(<span class="str">"invariants"</span>, ProcessedInvariants);
vtMethod.SetAttr(<span class="str">"requires"</span>, ProcessedRequires);
vtMethod.SetAttr(<span class="str">"ensures"</span>, ProcessedEnsures);
ArrayList beforeSnapshots =
SnapshotProcessor.GetBeforeSnapshots
(e.Method <span class="kwrd">as</span> MemberInfo, ProcessedSnapshotsBefore.Keys);
ArrayList afterSnapshots =
SnapshotProcessor.GetAfterSnapshots
(e.Method <span class="kwrd">as</span> MemberInfo, ProcessedSnapshotsAfter.Keys);
vtMethod.SetAttr(<span class="str">"ssbefore"</span>, beforeSnapshots);
vtMethod.SetAttr(<span class="str">"ssafter"</span>, afterSnapshots);
ProcessedMethods.Add(vtMethod.Merge());
ProcessedEnsures = <span class="kwrd">null</span>;
ProcessedRequires = <span class="kwrd">null</span>;
}
<span class="kwrd">public</span> <span class="kwrd">void</span> NewInvariantAttribute(<span class="kwrd">object</span> sender,
NewInvariantAttributeEventArgs e)
{
EnsureSpecification es = DbcPragmaProcessor.
ProcessEnsure(e.Invariant.Predicate);
SnapshotProcessor.RegisterSnapshots(es,
<span class="kwrd">ref</span> <span class="kwrd">this</span>.processedSnapshotsBefore,
<span class="kwrd"> ref</span> <span class="kwrd">this</span>.processedSnapshotsAfter);
vtInvariant.SetAttr(<span class="str">"invariant"</span>, es);
ProcessedInvariants.Add(vtInvariant.Merge());
}
<span class="kwrd">public</span> <span class="kwrd">void</span> NewEnsuresAttribute(<span class="kwrd">object</span> sender,
NewEnsuresAttributeEventArgs e)
{
EnsureSpecification es = DbcPragmaProcessor.
ProcessEnsure(e.Ensure.Predicate);
SnapshotProcessor.RegisterSnapshots(es,
<span class="kwrd">ref</span> <span class="kwrd">this</span>.processedSnapshotsBefore,
<span class="kwrd">ref</span> <span class="kwrd">this</span>.processedSnapshotsAfter);
vtEnsure.SetAttr(<span class="str">"ensure"</span>, es);
ProcessedEnsures.Add(vtEnsure.Merge());
}
}</pre>
<h3>Iterative vs final expansion of templates</h3><p>A seemlingly simple decision like the order in which you notify interested parties leads to significant performance issues that must be known about in advance. To illustrate this, consider the following scenario:</p><p><i style="FONT-SIZE: 11px">I want to use this framework in a number of different scenarios. I want to be able to statically generate the code for a proxy object that I then use as though it were the real object. That proxy object will enforce all of my rules at runtime. Projects that use my proxies must include the proxy assembly as well as the assembly of the target object. This is the simple case, I also need to be able to dispense with that and use dynamic proxies. Thus situation is one where I use dynamic code generation at runtime. To do that I need to request access to the object through a class factory that will perform code generation of the proxy on-the-fly. In other situations I want the framework to dovetail with aspect oriented programming frameworks like ACA.NET. In that situation the generated code could be either static or dynamic, but the specifics need to abstracted into a configuration file.</i></p><p>As you can see from these requirements our needs can swing between static and dynamic code generation, and we may even want to use both in the same program. Performance must take precedence if we want to use dynamic code generation. Static code generation won't suffer over much if we choose a code generation strategy that is biased towards dynamic code generation since its costs are at compile time, and won't affect runtime performance.</p>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1112077213091142232005-03-28T22:14:00.000-08:002005-04-12T17:35:45.830-07:00Code Generation Patterns - Part 1This time I'm going to show you the design pattern that I use for code generation. Generally I use this for analysis of hierarchical structures, such as assemblies or database metadata. You could use it in other scenarios, where data is less structured. If you are familiar with the operation of a SAX parser and .NET events then you'll have no trouble understanding the design.
<h4>SAX-like Parser</h4>A SAX parser works by iterating through its material (in this case XML, but that's immaterial) firing off events for everything that it finds. It doesn't do anything with what it finds, it just acts as an event generator that embodies some knowledge of the internal structure of the incoming material. It is up to whoever is listening to the events to do something with them. In the case of a SAX parser an XML DOM tree is probably built up, but not necessary, and that is the point. You have the option to tailor your processing of the sequence of events according to your requirements.
<h4>event scanner & generator</h4>In my design pattern the objective is to decouple event generation from event handling. As far as the event handler is concerned, it is the order of events and their content that matters, not where they originated from. Likewise for the scanner it is not important how the data is handled, only that it navigates through the data properly. So we decouple and encapsulate the two aspects of the process and enable the enhancement and extension of the system easily.
<h4>multicast events</h4>The use of events means that you can <i>multicast</i> events frm the scanner to as many handlers as you need.
<h4>typed events vs overloaded events</h4>A natural question that occurs to me when I liken the pattern I use for code generation with that of a SAX parser is why I use a plethora of events with typed parameters rather than an overloaded interface with a single event and event parameter type. With SAX events come in many shapes and sizes and are sent out through a single event interface. In my design I have chosen an interface and a set of events that represent each of the possible entities that we are interested in. The interface for the assembly scanner looks like this:
In a previous project I used the delegate/event interface as a common language between a set of components of an ORM system. The data initially made its way into the system through a reverse engineering database metadata schema reader. Apart from being a mouthful to say, it read the structure of a SQL Server database, and generated a set of events that were multicast in the ways I've described above. One of the listeners built an in-memory model of the schema. Later on we needed to persist this model to disk as a configuration file that would be used at runtime as the class map of the ORM (if this intrigues you, goto Scott Ambler's web site <++> for some very influential papers). So when the ORM service started up it loaded the configuration file and read through that firing off <i>exactly the same events</i>. The same listener was stationed on the other end of the event interface, so I was able to guarantee that the class map model was brought into memory in the same way as was used for the code generation cycle. My point is that this typed event interface acts as a common language between components participating in the data's lifecycle. If provided a natural depth first search model for passing data about in a way that allowed on the fly processing. It also allowed a complete decoupling of the data traversal mechanism from the data handling, the sequence of events was all that mattered to my code generation system, not the format on disk or even in memory - for all it cared the data could be stored as metadata in an RDBMS or nodes in an XML file or DOM. The decoupling is total, but not at the expense of vagueness as can be the case with catch all interfaces.
<pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">class</span> AssemblyScannerEventNotifier : MarshalByRefObject
{
<span class="kwrd">public</span> <span class="kwrd">delegate</span> <span class="kwrd">void</span> NewAssemblyHandler(<span class="kwrd">object</span> sender,
NewAssemblyEventArgs e);
<span class="kwrd">public</span> <span class="kwrd">event</span> NewAssemblyHandler newAssembly;
<span class="kwrd">public</span> <span class="kwrd">void</span> NotifyNewAssembly(Assembly assembly)
{
NewAssemblyEventArgs e = <span class="kwrd">new</span> NewAssemblyEventArgs(assembly);
<span class="kwrd">if</span> (newAssembly != <span class="kwrd">null</span>)
newAssembly(<span class="kwrd">this</span>, e);
}
}</pre>The EventArg object simply carries the assembly that has been found.
<pre class="csharpcode"><span class="kwrd">public</span> <span class="kwrd">class</span> NewAssemblyEventArgs : EventArgs
{
<span class="kwrd">private</span> Assembly theAssembly = <span class="kwrd">null</span>;
<span class="kwrd">public</span> Assembly TheAssembly
{
get{<span class="kwrd">return</span> theAssembly;}
}
<span class="kwrd">public</span> NewAssemblyEventArgs(Assembly assembly)
{
theAssembly = assembly;
}
}
</pre>What this allows is for us to multicast the event to a number of different interested parties. Generally that will be a small set including the code generator or some intermediate party, plus the test harness to provide a progress bar.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1111190693469198482005-03-18T15:53:00.000-08:002005-03-18T17:18:53.700-08:00Which codegen technique?Now that I've come up with a few requirements, and shown you how I am going to partition the problem, I think it might be about time to discuss some of our technological options.
I said previously that I had a serviceable NVelocity wrapper, that I could use for the code generation. I have other options too, though, and some of them might be better in the long run, than sticking with NVelocity. So I weighed them up, and found that there were many conflicting factors.
My codegen options are:
<ul><li>Write code to a text file using Stream.WriteLine</li><li>Templating language (such as NVelocity)</li><li>XSLT transformation</li><li>CodeDOM code emission</li>
</ul>I'm sure there are more options than these, but these are my main four. I judged them using the following criteria:
<ul><li>Speed</li><li>Ease of use</li><li>Flexibility</li><li>Readability</li><li>Debugability</li></ul>My <a href="http://aabs.aspxconnection.com/table1.html">comparison</a> is here. You can see that, as with most design decisions, there is a trade off to be made between speed and ease of development versus power and flexibility. With these sorts of decisions (especially when I know there are architectural factors involved) I tend to defer them until the last moment, or until something else places the casting vote. Of course, in the wider world of work, you normally get this sort of decision made for you by a pointy-haired marketing manager, who uses an etch-a-sketch as a laptop. Still, here I have the freedom to make my own decisions, and coolness and power will definitely win out over speed to market, or understandability.
Rant Over.
Apart from the CodeDOM option, I have used all of these techniques many times over the years. I've even used VIM macros to generated code. I've transformed XSD & DTD files into C++ serialization wrappers, and with Velocity and NVelocity I have done tons of code generation and generated literally millions of lines of code. The point is that the first three are tried and tested, with templates being my personal favourite. CodeDOM is not so dissimilar to systems I have used for HTML generation and XML tree navigation, so I guess it to is just another form of something familiar. Does that mean it doesn't win on the cool front? Fraid so. When it comes down to it, there are a few optimizations that can mitigate the startup, processing and resource management costs of using a template interpreter language such as NVelocity. The power and flexibility that we get in return is hard to refuse.
As you probably guessed I am going to opt for NVelocity, since I have a perfectly good system that I don't want to write again. I would like to have a framework to place it in that I can use with other code generation techniques if they seem worthwhile. Next time I will describe my code generation patterns and show how we can develop a limitless array of code generation handlers to allow people to use my system with XSLT or whatever. Later on I may even need to use this in conjunction with CodeDOM to perform dynamic proxy generation in a way similar to ServicedObjects in COM+.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1111097341786596212005-03-17T14:07:00.000-08:002005-03-18T15:06:49.910-08:00What to expect from a DBC framework<div>Last time I said I'd explore the requirements on a DBC solution in a little more detail. I first experienced design by contract when I was studying object oriented design at Brighton University, UK. I was studying it under Richard Mitchell, author of "Design by Contract, by Example". The curriculum was taught using Eiffel. Eiffel was (and is still is, as far as I know) the only programming language that has native support for design by contract. A little googling brings up quite a few links on DBC. One thing is clear from a cursory glance; DBC means different things to different people. To some it means prepending and appending your methods with guard statements like this:</div>
<pre class="csharpcode">
<span class="kwrd">public</span> <span class="kwrd">void</span> foo()
{
<span class="kwrd">if</span>(<span class="kwrd">this</span>.Bar < 1)
<span class="kwrd">throw</span> <span class="kwrd">new</span> ApplicationException(<span class="str">"Bar < 1 failed"</span>);
}
</pre>
<div>This is certainly a step in the right direction. It focuses your attention on what the environment of a component is like, and what the right "operating envelope" is for the correct function of a method. You do this when you define unit tests to explore the limits of a method, seeking ways to break it. Some argue that if you have done your job properly during the testing phase, then DBC wouldn't be necessary.</div>
<div>Others argue that such guard statements are against the tenets of defensive programming, but of course your part in the contract is to make sure that the code never reaches my component unless you are sure that the guards will not be triggered. So a defensive programmer would put guards around the method invocation. Naturally, I can't let my guards down, since as a developer I can't assume that you have done everything that you ought to do to use my component. So the checks are to stay in place, as a defensive measure against people who don't practice defensive programming themselves. Generally you can't handle argument errors at the point of invocation, so it makes sense to send the exception straight back to the caller.</div>
<div>Design By Contract is more than just a way of trapping and reporting errors in component usage. I want to be able to make a declaration like this on an interface, and have all implementers and users of the interface abide by the rules.</div>
<pre class="csharpcode">
[Invariant(Balance >= OverdraftLimit)]
<span class="kwrd">interface</span> IBankAccount
{
<span class="kwrd">public</span> <span class="kwrd">float</span> Balance{...}
<span class="kwrd">public</span> <span class="kwrd">float</span> OverdraftLimit{...}
}</pre>
<div>This sort of programming is more about making declarations about what you want to happen - declarative programming. It's about having declarations that have some weight. Sadly, in languages like C#, there is no intrinsic mechanism that enforces contracts like the one above, so the purpose of this project is to find a way to enforce such rules without intruding too much into the everyday routines of a programmer. It should just work.
</div>
<div>If you look closely at my interface definition above, you'll see that there is more going on. The interface is setting some semantic rules. Normally, an interface is a syntactic contract between component developers and consumers. But here we are able to go a step beyond that, into the realm of semantics If I produce a contract like this:</div>
<pre class="csharpcode">
<span class="kwrd">interface</span> IBankAccount
{
[Require(val > 0.0)]
[Ensure($after(Balance) == $before(Balance) + val)]
<span class="kwrd">public</span> <span class="kwrd">float</span> CreditAccount{<span class="kwrd">float</span> val}
<span class="kwrd">public</span> <span class="kwrd">float</span> Balance{...}
}</pre>
<div>I'm not just declaring how you make calls into my method. I'm telling you what the call <i>achieves</i>. Previously I was only able to say what the method was called, what parameters it took, what order they were in, and what type the result was. Vital stuff, to be sure, but not useful without a specification or some sort of API documentation. With DBC I can tell you what happens when you call my method, what kinds of data should be in the parameters, and what changes in the state data after the method call. I'm describing the effects of the process, without describing the algorithm. That's exactly what we're after with an interface. We have managed to extend the interface to carry the sort of information that it should have been carrying to begin with! Wow! Now you should be able to see why I'm writing a blog about DBC!</div>
<h4>In Summary...</h4>
<div>You can see from the examples above, that we need to be able to make statements about the content of Properties, Fields and Parameters. We also need to be able to take snapshots of their value before and after the method invocation, and see whether the rules have been followed. We now have some requirements to be going on with.</div>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1111010295085852042005-03-16T13:55:00.000-08:002005-03-16T14:41:44.476-08:00Domain Analysis (kinda)I'm blogging my progress from mid-way through the project. I have a working prototype, which is a crude implementation of what I want. It has a few limitations which require me to go back and productise it. It may seem a little arbitrary for me to describe a domain split on my second blog post, before even telling you what the requirements are. It's not. It's just that that was what I was thinking about, when I started this blog.
My motivation for this project is long standing. I used Eiffel at university (in 1993!), and since then have always wanted to see declarative predicates attached to interfaces and classes. Nothing ever came in the languages I use, so I always end up doing something like this. I wrote a version of this framework for C++ and Java some years ago, and I thought it's time I did the same for C#. I also want to explore some of the more obscure (and uniquely powerful) language features of C#, and new features coming with C# v2.0.
I've split up the broad levels of responsibility into the following:
1. Assertion testing.
2. Assertion representation
3. Assertion code generation
4. Assertion handler assembly management
5. Assertion failure management
6. Configuration
7. Third party code integration
Some of these areas I will automatically descope, on the basis that they are done better elsewhere. For example assertion failure management is done best using a top level exception management system, or handlers in the code from which the method invocations came. The Microsoft Enterprise System Exception Handler Application Block system will allow you to handle exceptions in a variety of ways, is extensible, and configurable. Lets leave that out. Assertion testing is probably just an if/then statement. But experience tells me that all result codes are not equal. HRESULTs needed a specialist test, as do old style C methods that have zero as a success code, and everything else as failure.
Configuration I have decided to offload onto the new Microsoft Configuration Application Block. Both came from the ACA.NET framework from Avanade. Kudos to them, and no bias intended at all from me.
Assertion representation is fairly simple. I have a set of Attributes for each type of assertion that I wish to make about a program. I have taken these from Eiffel: Invariant, Require and Ensure. These represent invariant assertions that must be true in all places and at all times. Ensure and Require are pre- and post-conditions that apply to whatever they are attached to only. These attributes store a string representation of the predicate they must enforce. It is up to this program to turn those string based predicates into lightweight code that can be applied on every invocation of the method they are attached to.
What I have already for code generation is a simple NVelocity based code generation template system. It is a dumbed down version of NVelocity, but has served me well over the years, and has been used in a production environment for ORM code generation. The system is simple enough to initialise in two lines, and allows repeated use of the same template with different parameters, so is very good for doing lots of code generation.
I currently use the CodeDOM framework for the compilation of the generated source code, and am as yet undecided about what to do with the generated assemblies. Should I save them to a DLL, and keep them around? Perhaps I could save myself the code generation step on future runs. I could also use a Just-In-Time assembly generator and add assertion handlers as they are encountered.
I am also undecided about whether to generate all of the code for the assembly inline as proxies, or use some sort of Layer Supertype pattern to outsource the assertion handling work, and then have a really barebones system emitted dynamically as MSIL to invoke method on the supertype as necessary.
Assertion handler assembly management is another area where I need to make some decisions. That will come out of how I solve the code generation issues described above.
Third party code integration is that part of the design process that makes sure that the framework is usable with a variety of implementations. For example I can imagine that this should work with Dynamic and Static proxies. Static are easier, but are more intrusive potentially. It should also fit into the more common Aspect Oriented Programming Systems. Again here I am inspired by Avanade's ACA.NET, which has a very well designed and implemented Aspect Oriented system, which I'm surprised is not in the enterprise Library Application Blocks.
So, to recap, I plan to write about how I solve the following problems in the coming weeks/months:
1. Assertion testing.
2. Assertion representation
3. Assertion code generation
4. Assertion handler assembly management
5. Third party code integration
I'll also describe some simple usage scenarios, to put all of this into context, which I guess will send me back to my university days and my first introduction to Eiffel. Eiffel is not a .NET compatible language, so you could say I am wasting my time re-inventing the wheel, when I could just program in Eiffel. My only excuse is that I first learned to program in C, and I bonded to the syntax. Anything else seems clunky or sloppy. I know that Eiffel is neither of these things, but I've never found a contract for Eiffel either, so I continue to trade on my C++, Java and C# skills.
There is another, better, reason: I'm working as a Solution Architect in Australia, where I don't get to write the programs I'm 'designing'. I'm doing this project in my private time. I need to keep my skills alive until I get a real project. So I'm not in any hurry to get this out, but if at some stage it gets robust enough to show the world, perhaps you would like to join me in a GPLd project? There are countless people out there who could do a better job of the third party framework integrations than me. Are you one of them? Are you single and have no triplets on the way? (unlike me!) Let me know, and when the time comes, I'll set up the project.aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0tag:blogger.com,1999:blog-11473797.post-1110924458273253462005-03-15T13:51:00.000-08:002005-04-12T17:33:10.263-07:00What I'm gonna cover<span style="font-size:85%;"><span style="font-family:arial;">When I get the time I intend to start off this blog by exploring the following ideas:
</span></span> <ul> <li><span style="font-size:85%;"><span style="font-family:arial;">Developing a design by contract system for C#</span></span></li> <li><span style="font-size:85%;"><span style="font-family:arial;">History of Design by Contract</span></span></li> <li><span style="font-size:85%;"><span style="font-family:arial;">Code Generation Systems XSLT vs NVelocity</span></span></li> </ul> <span style="font-size:85%;"><span style="font-family:arial;">I'm producing a Design by contract system that's in early alpha. It uses NVelocity, and declarative programming. It lets you write code like this:
<!-- code formatted by http://manoli.net/csharpformat/ -->
<pre class="csharpcode">
[Dbc, Invariant(<span class="str">"Prop1 >= 1"</span>)]
<span class="kwrd">public</span> <span class="kwrd">class</span> MyTestClass
{
<span class="kwrd">public</span> MyTestClass()
{
Prop1 = 10;
}
<span class="preproc">#region</span> property Prop1
<span class="kwrd">int</span> prop1 = 0;
[Requires(<span class="str">"value != 1561"</span>)]
<span class="kwrd">public</span> <span class="kwrd">int</span> Prop1
{
get
{
<span class="kwrd">return</span> prop1;
}
set
{
prop1 = value;
}
}
<span class="preproc">#endregion</span>
[Requires(<span class="str">"arg1 > 10"</span>)]
[Requires(<span class="str">"arg2 < 100"</span>)]
[Ensures(<span class="str">"$after(Prop1) == $before(Prop1) + 1"</span>)]
<span class="kwrd">public</span> <span class="kwrd">void</span> TestMethod(<span class="kwrd">int</span> arg1, <span class="kwrd">int</span> arg2, <span class="kwrd">string</span> arg3)
{
Prop1 = Prop1 + 1;
System.Diagnostics.Debug.WriteLine(
<span class="str">"MyTestClass.TestMethod.Prop1 == "</span> + prop1.ToString());
}
}</pre>
During the rest of the project I'll show you how I expand this system to work with Aspect Oriented Programming, and Static and Dynamic Proxies. I'll also explore the best ways of implementing the control logic under the hood.
Till then, I guess we're stuck with Asserts or ifs at the beginning of methods.
</span></span>aabshttp://www.blogger.com/profile/00137991688197838165noreply@blogger.com0