Turbo-Charge Your Report Speed - General Rules & Guidelines

Product

One advantage as the CTO (and creator) at Windward Studios is I have seen just about anything & everything someone can do with reports, document generation, dashboards, etc. And with Windward being free-form, I've probably seen more variations than my opposite number at our competitors. From this perspective I've learned a number of basic do's and don'ts that I think hold across all reporting, docgen, & dashboard systems.

Report Design

If nothing else follow this rule – do not fight your report designer. If you use a system that assumes banded reports (Crystal, SSRS, etc.) then create a banded report. If you try to work at cross-purposes you will find it difficult, frustrating, and the final design will be a series of compromises.
Use the appropriate approach. For example with Windward you can design in Word or Excel (or PowerPoint). Yet we have customers designing spreadsheets using Word tables and designing documents with a couple of tables in Excel. When they switch from Word to Excel (or vice-versa), then something that was hard becomes very easy.
Understand and accept the limitations of your designer. For example, SSRS treats sub-reports as separate reports so you cannot render a sub-report in the context of the report pulling it in. You need to keep this in mind when architecting your template structure.
K.I.S.S. – Keep It Simple (Stupid). I've had numerous cases where, when diving into the issues a customer had, the core issue was they were not clear about what information they were trying to get across, and what data represented that information. Try to be very clear on what you are presenting and what that information is.
Pie charts are evil.

System Architecture

If the reporting engine runs in the context of your application, create multiple threads for multiple simultaneous report requests. Report generation tends to be heavily I/O bound so you get a big win here. Our rule of thumb is have twice as many threads as you have cores on your system. (And Hyper-threading counts as distinct cores for this measure.)
Buy sufficient memory. If you are generating a 10,000 page report, you need more than ½Gig.
Watch your dependence on network I/O. If you are pulling in images or sub-reports over the network, that network connection can easily be the gating factor in your speed.
Run a profiler against your system under load. Generally the largest time hit will be a surprise, often one you can address.

Data Access

Only pull in the data you need. A SQL select should almost never start with "select * …" because you rarely need every column. Select only the columns you need to display in the report.
You do not need to select columns you use in your joins, order by, or where criteria (I see this a lot). You may use columns in the select without returning them.
For a complex join, add it to the database as a view if possible. If that cannot be done, create it as a dataset. A database is optimized to handle complex joins as a view and most reporting systems will build a dataset as a temporary view.
For XML data, minimize the size of the XML dataset. Using XPath or XQuery requires that the entire XML structure be read in to memory as a DOM or XPath optimized structure. A larger dataset means larger memory usage and more nodes to traverse on a query. (For Windward we've found several hundred megabytes is usually fast, but when you get into gigabytes it starts to have an impact.)
Don't pull data twice. If you are iterating through rows of data (or nodes in XML), then for each iteration pull all the needed data for that row/node in the main iterator and then access it directly. It should be rare that you have a select statement inside an iterative loop.
For SQL make sure all columns used for joins and the key parts of a sort or where clause are indexed. (Most people will say all, and disk space is cheap. But if you always sort by last_name, first_name then while last_name must be indexed, first_name should be indexed if space allows. Life is trade-offs.)
For XML create a schema with data typing. This avoids ambiguity where XPath/XQuery not only have to make assumptions, but they have to do extra processing determining what they should assume.

Vendor Best Practices

‍

If you've just discovered us, we're excited. Know more about Windward and get your 14-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Tags Start & End

Tags Can Start & End Anywhere

Appendix B

.NET code for multi-page image output

Appendix A

Java code for multi-page image output

Data Bin Search

The Data Bin can now be searched to find a table, column, node or other piece of data without scrolling through it all.

Shrink to Fit

This will shrink the contents of a cell until it fits the defined cell size.

Time Zone Conversion

A new Windward macro has been added to help with converting dates and times from UTC time to the local time zone.

Image Output Format

New image output formats added.

PostScript Output Format

PostScript, commonly used with printers and printing companies, has been added as an additional output format.

New and Improved Datasets (Designer, Java Engine, .NET Engine)

Datasets have been re-written from scratch to be more powerful and easier to use.

Stored Procedure Wizard (Designer)

This works for all tag types that are connected to a SQL-based data source (Microsoft SQL Server, Oracle, MySQL, or DB2).

Boolean Conditional Wizard (Designer)

Before, conditional statements could only be written manually. Now they can also be built using our intuitive Wizard interface.

Reorganized Ribbon

The ribbon menus have been re-organized and consolidated to improve the report design workflow.

XPath 2.0 as Data Source

Adds various capabilities such as inequalities,descending sort, joins, and other functions.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

SQL Select Debugger

The look and feel was improved
Stored Procedure Wizard
Improved Exceptions pane

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Tag Editor/Tag Selector

Added a Query tab as a field for typing or pasting in a select statement

Color Coding of Keywords
TypeAhead
Evaluate is now "Preview"

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Rename a Datasource

All tags using that Data source will be automatically updated with that name.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Connecting to a Data Source

New single interface to replace 2 separate dialog boxes

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Tag Tree

Displays of all the tags in the template, structured as they are placed in the template. This provides a simple & intuitive way to see the structure of your template. Also provides the capability to go to any tag and/or see the properties of any tag.

Added Javelin into the RESTful Engine

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Support for Google Application Engine Integration

The ability to integrate the Windward Engine into Google’s cloud computing platform for developing and hosting web applications dubbed Google Applications Engine (GAE).

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Additional Refinement for HTML Output

Improved indentation for ordered and unordered lists
Better handling of template header and footer images
Better handling for background images and colors

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Redesigned PDF Output Support

This new integration will allow for processing of complex scripts and bi-directional text such as Arabic. Your PDF output will be much tighter and more closely match your template, and we’ll be able to respond rapidly to PDF requests and fixes.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

PowerPoint Support

Includes support for new ForEach and slide break handling, table header row repeat across slide breaks, and native Microsoft support for charts and images.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Tags are Color Coded

Tags are color coded in the template by type, making it easy to visually identify them.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Increased Performance

Version 13’s core code has been reworked and optimized to offer a reduced memory footprint, faster PDF generation and full documentation of supported features and limitations in the specifications for DOCX, XLSX and PPTX.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Advanced Image Properties

Documents can include advanced Word image properties such as shadows, borders, and styles.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Improved HTML Output

Windward has updated HTML output to reflect changing HTML standards.

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Version 13 New Data Sources

Windward now works with a slew of new datasources: MongoDB, JSON, Cassandra, OData, Salesforce.com

‍

If you've just discovered us, we're excited. Learn more about Windward document automation software now.

‍

Try Windward with our 30-day free trial and start creating documents in quick time with our low/no code solutions.

‍

Generate Code

The Generate Code tool in the designer allows you to open an existing template and, with a click of a button, automatically create a window with the code needed to run your current template with all data sources and variables. Simply copy this code and paste into your application's code in the appropriate place. You now have Windward integrated into your application.

You only need to do this once. You do not do this for each template. Instead, where it has explicit files for the template and output, change that to parameters you pass to this code. Same for the parameters passed to Windward. This example uses explicit values to show you what to substitute in where.

Pivot Tables Adjusted in Output

Any pivot tables in an XLSX template are carried over to the XLSX output. The ranges in the pivot ranges are adjusted to match the generated output. So your final XLSX will have pivot tables set as expected in the generated file.

This makes creating an XLSX workbook with pivot tables trivial.

Imported Template Can be Set to Match the Parent Styles

In an imported sub-template, if its properties for a style (ex. Normal) differ from the parent template's properties for the style, the use in the sub-template can be set to either use the properties in the sub-template, or the properties in the parent.

You set to retain when you don't want the child template's styling to change when imported. You set to use the parent when you want the styling of the imported template to match the styling in the parent.

Any explicit styling is always retained. This only impacts styling set by styles.

Tags can be Placed in Text Boxes

Tags can be placed in text boxes. Including linked text boxes. This gives you the ability to set the text in a textbox from your data.

Tags can be Placed in Shapes & Smart Art

Tags can be placed in shapes & smart art. This gives you the ability to set the text in a shape from your data.

HTML Output Supports Embedded Images

When generating HTML output, the engine can either write bitmaps as distinct files the generate HTML references, or it can embed the images in the HTML providing a single file for the output.

Footnotes & Endnotes can Have Tags

You can place tags in pretty much any part of a template, including in footnotes & endnotes.

Document Locking Supported in DOCX & XLSX

Any parts of a DOCX or XLSX (PowerPoint does not support this) file that are locked in the template, will be locked the same in the output.

Specify Font Substitution

If a font used in the template does not exist on the server generating a report, the font to substitute can be specified.
In addition, if a glyph to be rendered does not exist in the font specified, you can specify the replacement font. This can be set distinctly for European, Bi-Directional, and Far East fonts.

Process Multiple Datasources Simultaneously

If you need this - it's essential. And if you don't need it, it's irrelevant.

Windward enables you to build a document by applying multiple datasources to the template simultaneously. When Windward is merging the data into a template, it processes the template by handling each tag in order, and each tag pulls from different datasources. (As opposed to processing all of one datasource, then processing the next.)

This allows the select tag to use data from another datasource in its select. For example, if you are pulling customer information from one data source, you can then pull data from the sales datasource using the customer ID of the customer presently processing to pull the sales information for that customer. If you're interested in patching together your data from multiple datasources, read this post on our blog.

Written by:_

David Thielen

President/CEO at Windward Studios

Turbo-Charge Your Report Speed - General Rules & Guidelines

Report Design

System Architecture

Data Access

Vendor Best Practices

Tags Start & End

Appendix B

Appendix A

Data Bin Search

Shrink to Fit

Time Zone Conversion

Image Output Format

PostScript Output Format

New and Improved Datasets (Designer, Java Engine, .NET Engine)

Stored Procedure Wizard (Designer)

Boolean Conditional Wizard (Designer)

Reorganized Ribbon

XPath 2.0 as Data Source

SQL Select Debugger

Tag Editor/Tag Selector

Rename a Datasource

Connecting to a Data Source

Tag Tree

Added Javelin into the RESTful Engine

Support for Google Application Engine Integration

Additional Refinement for HTML Output

Redesigned PDF Output Support

PowerPoint Support

Tags are Color Coded

Increased Performance

Advanced Image Properties

Improved HTML Output

Version 13 New Data Sources

Generate Code

Pivot Tables Adjusted in Output

Imported Template Can be Set to Match the Parent Styles

Tags can be Placed in Text Boxes

Tags can be Placed in Shapes & Smart Art

HTML Output Supports Embedded Images

Footnotes & Endnotes can Have Tags

Document Locking Supported in DOCX & XLSX

Specify Font Substitution

Process Multiple Datasources Simultaneously

Get tips straight to your inbox and become a better document creator.