Exporting results to binary files

InfoWorks ICM allows you to export results in various formats. Amongst all the options available you can export results to binary files. Have you considered using this format? Continue reading this blog post to find more details about this.

What is a binary file?

A binary file stores data in binary format which can be read by computers but is not readable by users. It is usually more efficient than other formats, because their size is generally much smaller and their input and output is much faster.

Why do we want a more efficient file format?

We want a more efficient file format to be more efficient! This is particularly important if gigabytes of results are being generate and analysed. For example, the Stochastic Flood Maps presented in this blog post required the analysis of runs with 200 simulations, each of them with more than 10 000 2D elements, which ends in the generation and analysing of 1.2 GB of results for each run. Under these circumstances, the size of results files exported and the time required for their analysis can be significantly reduced with the use of binary files.

This does not mean the binary format should always be used. The use of binary files require the development of third party applications to read them, which might not be convenient for common modelling tasks that do not involve much data. In this case, the use of other formats, such as csv files, can be much handier to use and easier to read and analyse. In addition, InfoWorks ICM has several tools that can be used for most modelling tasks, including displaying and analysing results.

How to export to binary files in InfoWorks ICM?

InfoWorks ICM has the capability of exporting results to binary files from the user interface and with InfoWorks ICMExchange which is the programmer’s toolkit of InfoWorks ICM.

In the user interface, results can be exported to binary files using the “Binary Export” tool shown in Figure 1.

Using InfoWorks ICMExchange, the expression “results_binary_export” can be used in a ruby script to export results to binary files.

Figure 1. “Export to binary files” tool in InfoWorks ICM.

Is this efficiency really true?

To demonstrate this, here is the conclusion of a test conducted in InfoWorks ICM and ICMExchange:

  • Exporting times reduced to nearly 40%: In the user interface, the time required to export the binary files was 78% of the time required to export to csv files. Using InfoWorks ICMExchange, this time was reduced to 61% of the time required to export to csv files.
  • Size of exported files reduced to more than 90%: The size of the binary files was 8% of size with the same results in csv files.

The test summarized was based on exporting all results attributes of one node and one link (of a 1D network) for 31 simulations with 3h duration and results every 60s. The ruby script used in InfoWorks ICMExchange is the following:

db=WSApplication.open ‘snumbat://localhost:40000/db_75’,false #dataserver:port/database_name

i=1 #id of the first sim

sel=db.model_object_from_type_and_id(‘Selection list’, 10699)

while i<=31 #id of the last sim

sim=db.model_object_from_type_and_id(‘Sim’, i)

sim.results_binary_export(sel, nil , ‘C:\temp\sim’ + i.to_s + ‘.dat’)

i+=1

end

puts “Well done!”

And how can I read binary files?

The binary files are intended to be read by programs specially written by user organisations or by third parties to read them rather than off the shelf software such as spreadsheets, databases etc. If you are interested in further details about this, please contact our Technical Support team: support@innovyze.com

Share this post!

    About Rui Daniel Pina

    Rui Daniel Pina is a Technical Support Engineer with Innovyze in the United Kingdom. Rui has experience in hydrology and hydraulic modelling, and is responsible for providing support of Infoworks ICM.
    This entry was posted in InfoWorks ICM. Bookmark the permalink.