I've found that a lot of people have a better understanding of our system when I explain how the engine processes a report. (Others are very content with "it works.")
First off, the engine does not use Microsoft Office. You design in Office but the server itself does not use Office. And when in AutoTag you generate a report, that generate process uses the .net reporting engine and does not make use of Office, it is merely a self-contained app being run from AutoTag. So you do not install Office on your server (calling Office to generate reports would be a very slow system and would only run on Windows – that's why we don't do it).
The Java engine is pure Java and so will run on any system that has the Java virtual machine. We have customers using us on various flavors of Linux, AIX, Mac, and virtually every IBM mainframe. The .NET engine is pure .NET – 100% managed code. It does not use Java and does not require a Java virtual machine.
We do this with a single code base so that both engines are the same program and we only have to write code once. How do we do this? For the .NET version we use the Microsoft J# compiler which compiles Java source code into .NET CLR (which is the same thing the C# compiler does). About 2% of the code is specific to each platform and the rest is the same code, compiled twice, by the javac and devenv compilers. The 2% that is different is a set of classes where the public class definition is identical for both platforms, but the code in each method is unique to that platform. We use this to wrap the calls for XML manipulation, bitmap generation, etc.
There are three substantial differences between the Java and .NET reporting engines. And keep in mind that AutoTag uses the .NET engine to show you a sample report even if your server uses the Java engine.
- The XML datasources are completely different. The Java engine uses dom4j & jaxen and the .NET engine uses the .NET System.Xml classes in the runtime. These work identically except for some corner cases where the XPath spec is ambiguous. My take on the differences is jaxen generally does what appears to be the most sensible interpretation while .NET does what is the most correct interpretation (but it takes a lot of explanation to see that correctness). But the end result is it rarely matters.
- The SQL datasources are also completely different. The Java engine uses JDBC and the .NET engine uses ADO.NET connectors. However, SQL is so mature and so well defined that there is no difference in the result.
- We use different charting engines for the Java and .NET versions. Charts will look somewhat different. And some of the more esoteric charting settings are only supported on one or the other engine. Both are very pretty, but they are different. (And the gauges are very different – and not the prettiest with the Java engine.)
Ok, now we know how it is built and installed. Next is how the code generates a report. This is pretty straightforward:
- The template is parsed (for the OpenXML formats this uses a SAX parser to walk the elements in each of the template's individual xml files). This builds the template in memory as a Document object. This internal object is format neutral although for every format, some properties in the internal object can only be populated for a specific template type. For example, only an Excel template has the concept of conditional cell formatting and so only for an Excel template will that property be set.
Note: PPTX output requires so many unique properties that you can only create a PPTX report from a PPTX template.
- We then use the visitor pattern to break out the windward tags within the template and perform additional preparation of the template. For example, we mark if any of the document has right-to-left text (Hebrew or Arabic) as that requires additional processing.
Note: parts 1 & 2 are what occurs in processSetup().
- We next use the visitor pattern to apply a datasource to the template. We do this one datasource at a time, the engine evaluating and applying each tag as it hits it in the template. Any tags that are for another datasource are copied across as is. You can apply as many datasources as you wish. And if you apply a datasource that the template has no tags for, it has no effect on the report.
- The final step is to lay out the final report. With the previous effort we have the full report as a series of paragraphs and tables, but it needs to be laid out on the page, adding soft line and page breaks and determining the position of every part of the document on the page. This is fully used for PDF, TXT, and printer output where we must draw the report on each page. But parts of it are needed for every output format. For example, if you have a Table of Contents in a Word document, we have to write out the page number of each chapter in the TOC field – Word does not recalculate this when opening a document.
This part is probably the most complex component of the engine. First off, the rules are complex, very very complex. Second, you have to lay out a page to see if that layout is acceptable. For example, if you have a series of paragraphs marked keep together and keep with next, you need to lay out all of those paragraphs to see if they fit on the present page. If they do not, you have to jump back, insert a soft page break, then start that series of paragraphs on a new page.
Note: part 4 is what occurs in processComplete().
So there you have it. The concept is simple, but the effort under the covers is significant. I look back at how little we handled in version 1.0 vs. what we do now and I'm amazed at the breadth of properties that must be taken into account and the literally thousands of special cases, rules, etc. we have learned about and added to our code.
But the core API of the server has never changed, and I take a lot of pride that we've kept it simple through 10 versions. And we've kept installing it on a server equally simple – just copy the files onto the server and call them from your application.