< Back to Blog

Turbo-Charge Your Report Speed - General Rules & Guidelines

Product

One advantage as the CTO (and creator) at Windward Studios is I have seen just about anything & everything someone can do with reports, document generation, dashboards, etc. And with Windward being free-form, I've probably seen more variations than my opposite number at our competitors. From this perspective I've learned a number of basic do's and don'ts that I think hold across all reporting, docgen, & dashboard systems.

Report Design

  1. If nothing else follow this rule – do not fight your report designer. If you use a system that assumes banded reports (Crystal, SSRS, etc.) then create a banded report. If you try to work at cross-purposes you will find it difficult, frustrating, and the final design will be a series of compromises.
  2. Use the appropriate approach. For example with Windward you can design in Word or Excel (or PowerPoint). Yet we have customers designing spreadsheets using Word tables and designing documents with a couple of tables in Excel. When they switch from Word to Excel (or vice-versa), then something that was hard becomes very easy.
  3. Understand and accept the limitations of your designer. For example, SSRS treats sub-reports as separate reports so you cannot render a sub-report in the context of the report pulling it in. You need to keep this in mind when architecting your template structure.
  4. K.I.S.S. – Keep It Simple (Stupid). I've had numerous cases where, when diving into the issues a customer had, the core issue was they were not clear about what information they were trying to get across, and what data represented that information. Try to be very clear on what you are presenting and what that information is.
  5. Pie charts are evil.

System Architecture

  1. If the reporting engine runs in the context of your application, create multiple threads for multiple simultaneous report requests. Report generation tends to be heavily I/O bound so you get a big win here. Our rule of thumb is have twice as many threads as you have cores on your system. (And Hyper-threading counts as distinct cores for this measure.)
  2. Buy sufficient memory. If you are generating a 10,000 page report, you need more than ½Gig.
  3. Watch your dependence on network I/O. If you are pulling in images or sub-reports over the network, that network connection can easily be the gating factor in your speed.
  4. Run a profiler against your system under load. Generally the largest time hit will be a surprise, often one you can address.

Data Access

  1. Only pull in the data you need. A SQL select should almost never start with "select * …" because you rarely need every column. Select only the columns you need to display in the report.
  2. You do not need to select columns you use in your joins, order by, or where criteria (I see this a lot). You may use columns in the select without returning them.
  3. For a complex join, add it to the database as a view if possible. If that cannot be done, create it as a dataset. A database is optimized to handle complex joins as a view and most reporting systems will build a dataset as a temporary view.
  4. For XML data, minimize the size of the XML dataset. Using XPath or XQuery requires that the entire XML structure be read in to memory as a DOM or XPath optimized structure. A larger dataset means larger memory usage and more nodes to traverse on a query. (For Windward we've found several hundred megabytes is usually fast, but when you get into gigabytes it starts to have an impact.)
  5. Don't pull data twice. If you are iterating through rows of data (or nodes in XML), then for each iteration pull all the needed data for that row/node in the main iterator and then access it directly. It should be rare that you have a select statement inside an iterative loop.
  6. For SQL make sure all columns used for joins and the key parts of a sort or where clause are indexed. (Most people will say all, and disk space is cheap. But if you always sort by last_name, first_name then while last_name must be indexed, first_name should be indexed if space allows. Life is trade-offs.)
  7. For XML create a schema with data typing. This avoids ambiguity where XPath/XQuery not only have to make assumptions, but they have to do extra processing determining what they should assume.

Vendor Best Practices

  1. BIRT Best Practices (more why you need a 3 day class to learn all of them)
  2. Crystal Reports Best Practices
  3. Microsoft Sql Server Best Practices
  4. Oracle B.I. Publisher Best Practices
  5. Windward Reports Best Practices
  6. General tips on designing a report

Tags Start & End

Tags Can Start & End Anywhere

Appendix B

.NET code for multi-page image output

Appendix A

Java code for multi-page image output

Data Bin Search

The Data Bin can now be searched to find a table, column, node or other piece of data without scrolling through it all.

Shrink to Fit

This will shrink the contents of a cell until it fits the defined cell size.

Time Zone Conversion

A new Windward macro has been added to help with converting dates and times from UTC time to the local time zone.

Image Output Format

New image output formats added.

PostScript Output Format

PostScript, commonly used with printers and printing companies, has been added as an additional output format.

New and Improved Datasets (Designer, Java Engine, .NET Engine)

Datasets have been re-written from scratch to be more powerful and easier to use.

Stored Procedure Wizard (Designer)

This works for all tag types that are connected to a SQL-based data source (Microsoft SQL Server, Oracle, MySQL, or DB2).

Boolean Conditional Wizard (Designer)

Before, conditional statements could only be written manually. Now they can also be built using our intuitive Wizard interface.

Reorganized Ribbon

The ribbon menus have been re-organized and consolidated to improve the report design workflow.

XPath 2.0 as Data Source

Adds various capabilities such as inequalities,descending sort, joins, and other functions.

SQL Select Debugger

SQL Select  Debugger

  • The look and feel was improved
  • Stored Procedure Wizard
  • Improved Exceptions pane

Tag Editor/Tag Selector

Added a Query tab as a field for typing or pasting in a select statement

  • Color Coding of Keywords
  • TypeAhead
  • Evaluate is now "Preview"

Rename a Datasource

All tags using that Data source will be automatically updated with that name.

Connecting to a Data Source

New single interface to replace 2 separate dialog boxes

Tag Tree

Provides a display of all the tags in the template, structures as they are placed in the template. Provides an easy way to see the structure, go to any tag, and see the properties of a tag.

Added Javelin into the RESTful Engine

Support for Google Application Engine Integration

The ability to integrate the Windward Engine into Google’s cloud computing platform for developing and hosting web applications dubbed Google Applications Engine (GAE).

Additional Refinement for HTML Output

  • Improved indentation for ordered and unordered lists
  • Better handling of template header and footer images
  • Better handling for background images and colors

Redesigned PDF Output Support

This new  integration will allow for processing of complex scripts and bi-directional  text such as Arabic.  Your PDF output  will be much tighter and more closely match your template, and we’ll be able  to respond rapidly to PDF requests and fixes.

PowerPoint Support

Includes support for new ForEach and slide break handling, table header row repeat across slide breaks, and native Microsoft support for charts and images.

Tags are Color Coded

Tags are color coded in the template by type, making it easy to visually identify them.

Increased Performance

Version 13’s core code has been reworked and optimized to offer a reduced memory footprint, faster PDF generation and full documentation of supported features and limitations in the specifications for DOCX, XLSX and PPTX.

Advanced Image Properties

Documents can include advanced Word image properties such as shadows, borders, and styles.

Improved HTML Output

Windward has updated HTML output to reflect changing HTML standards.

Version 13 New Data Sources

Windward now works with a slew of new datasources: MongoDB, JSON, Cassandra, OData, Salesforce.com

Generate Code

Designer (fka AutoTag) Generate Code tool allows you to open an existing template and, with a click of a button, automatically create a window with the code needed to run your current template with all data sources and variables in your .NET or Java project. The process is quick, shortens delivery time and helps alleviate any bumps in the road.

Pivot Tables Adjusted in Output

Any pivot tables in the template are carried over to XLSX output. The ranges in the pivot ranges are adjusted to match the generated output. So your final XLSX will be pivot tables set as expected in the generated file.

Imported Template Can be Set to Match the Parent Styles

In an imported sub-template, if its properties for a style (ex. Normal) differ from the parent template's properties for the style, the use in the sub-template can be set to either use the properties in the sub-template, ot the properties in the parent.

Tags can be Placed in Text Boxes

Tags can be placed in text boxes. Including linked text boxes. This gives you the ability to set the text in a textbox from your data.

Tags can be Placed in Shapes & Smart Art

Tags can be placed in shapes & smart art. This gives you the ability to set the text in a shape from your data.

HTML Output Supports Embedded Images

When generating HTML output, the engine can either write bitmaps as distinct files the generate HTML references, or it can embed the images in the HTML providing a single file for the output.

Footnotes & Endnotes can Have Tags

You can place tags in pretty much any part of a template, including in footnotes & endnotes.

Document Locking Supported in DOCX & XLSX

Any parts of a DOCX or XLSX (PowerPoint does not support this) file that are locked in the template, will be locked the same in the output.

Specify Font Substitution

If a font used in the template does not exist on the server generating a report, the font to substitute can be specified.
In addition, if a glyph to be rendered does not exist in the font specified, you can specify the replacement font. This can be set distinctly for European, Bi-Directional, and Far East fonts.

Process Multiple Datasources Simultaneously

When Windward is merging the data into a template, it process the template handling each tag in order, and the tags can pull from different datasources. This allows the select in a tag to use data from other datasources in the select.

David Thielen

President/CEO at Windward Studios

From his early years as a Senior Developer at Microsoft, to legendary designer of the popular Enemy Nations strategy game, to reporting and document generation guru, Dave has never lost his passion for building superb software and teams.

david@windward.nethttps://www.linkedin.com/in/davethielen/

For over 10 years, Windward has lead the industry with our world-class document generation platform that creates visually stunning, data-powered documents designed exactly the way users want and are created in a fraction of the time and cost compared to existing solutions. Proudly located in Boulder, Colorado, Windward Studios is the premier solution for developers and business users adding reporting and document generation capabilities to their applications in over 70 countries around the world.

Contact

Got questions about reporting and document generation? We've got answers—let's connect!
Send a note