The way you compute computational fluid dynamics (CFD) problems on Rescale is very similar to running on your local environment. Some of these tips may be obvious to those familiar with CFD but here are a few suggestions, in no particular order, for running CFD on Rescale.

Garbage in, garbage out

It goes without saying that a good mesh could make or break your run. But before focusing on the mesh quality, one should always ensure that the job they wish to submit to Rescale should run end-to-end locally. Occasionally, there are users who use hundreds of cores expecting it to run for many days, only see it “finish” in a few seconds. We all make mistakes, but it is probably not productive to repeatedly make small mistakes that have a big cost. Executing a small case of what you will be running locally will save your time and your wallet.

Watch your step

The Rescale platform has a Live Tailing feature that allows you to view all the files while the computation is in progress. It is helpful to know if there are errors or the progress it has made. However, in CFD you are also interested in knowing the residuals and certain values as the simulation progresses.  One way to monitor your iteration is writing out the plot as image file(s).
The figure below is an example of a residual output from STAR-CCM+ running a steady-state problem.


This residual plot is written out as a jpeg file with the code shown below. You will probably need to make changes to this code to integrate with your code.


Alternatively, you could plot your own figure with the built-in-tools. For software that writes the residual log as ASCII file, one method to plot the residual is by using gnuplot.


Below is a code for gnuplot to write a jpeg image file of the residual plot. One should modify the example code below to suit their own CFD software output.


Guide your run

Sometimes it is necessary to intervene during the computation and change some settings, such as relaxation factor, save interval, or solver parameter. One can ssh into the compute cluster having set an ssh key in their account settings. For details, see here. Intervention may help save a run if one identifies bad parameter choice(s), thus avoiding submitting a new job. Of course to know if you need to intervene, you will need to be watching your step.
One could also run a non-graphical interactive session if the CFD software supports it. Below is a screenshot of the terminal session of ANSYS Fluent.


For hoarders – package files

Result files are precious, especially with all the work that went into just getting it to run. Some software writes out many files of relatively small size (less than few MB). With thousands of these files, the data transfer time after the computation completes can become a large portion of the job duration. This data transfer applies both to when the results are transferred to the storage space as well as when you download the results to your local workstation at a later time. The large number of files and large file size might not be avoidable in CFD, however, packing up many files and writing in binary can mitigate the time spent on going for a coffee break.

As an example, OpenFOAM writes all its field solutions in separate files (e.g U, p, T, … etc.). This is convenient for post-processing or using the output for another input, but creates many files. One could write in binary and limit the write interval in the system/controlDict setting. Additionally, packaging up the results with tar or zip is strongly advised, but then again… coffee might be calling.

Looking into the Rescale cloud

Result files too large for your bandwidth aren’t a problem anymore. You can view them without downloading the results. Simply start a remote desktop and load your job that is in progress or a completed job. Shown below is a Windows remote desktop session (Linux is also available).

Another option is to start ParaView on the job cluster and use ParaView server to view the results. Compared to the remote desktop, this is limited to the operations of ParaView. If all you wish to do is view and extract certain data, this may be all you need. For more details see here.



This article was written by Hiraku Nakamura.

Lately, news from West Africa is almost always on Ebola. This is not a new problem, however, this historical Ebola epidemic continues to have a significant impact in the region. At the very least, you will see what looks like a tree with sparkling lights (see the videos below). In actuality, the following is an ab initio model of the disease spreading in a hypothetical network. This is not a blog about how Ebola will take over the world, but perhaps it will hopefully give you an insight as to why it will not.


The limited locality and the number of cases reported makes Ebola a disease requiring estimates that likely do not reflect reality. With limited information available we take a direct approach by simulating the person-to-person and person-to-public interaction. The model consists of five stages.

The stages of Ebola are: not exposed, infected, contagious, undercare, and deceased or recovered. Everyone except patient(s) zero is initially not exposed to the virus. The original patient(s) progress through the stages and eventually become contagious. We simulate a case where 5 people randomly selected are infected. The incubation period is normally distributed with a mean of 19 days. Followed by 3 days of the contagious period and finally 10 days of medical care. These two periods are both normally distributed about their respective means.

There are two infection paths. The first is the relationships between those we know. The second is through the public space. A person can only become infected through both of these paths and when they interact with a person who is contagious. A person who is contagious eventually becomes too sick and goes under medical quarantine. At this point, the patient is isolated from society. We also assume that the number of medical staff who become infected is insignificant.

Beyond the simplifications described, there are no control measures to counter the spread of Ebola except the quarantine imposed on the sick. The parameters used are at best, estimates, and likely do not reflect reality. We set the probability of infection by interacting with a contagious person at 50% and the infection by interacting in public at 1%. The probability of death is 60%. Below is a video of this simulation with 100,000 people in a hypothetical network for 300 days. The stages are colored, with white representing a healthy person, yellow is infected, red is contagious, orange is quarantined, black represents death and blue represents recovery.

It is apparent from the video that the propagation of waves originates from the major cluster in the network.  We can see these manifested in the figure below. Without the video above, the oscillation in the figure below can be difficult to understand. This was a simple model, but it should be clear that quarantine is a crucial and an effective measure against the spread the virus. Below in Graph 1 is of the simulated model representing the cases that became infected.


Graph 1: Simulated model representing the Ebola cases that became infected.

Influenza vs. Ebola

Arguably, what should concern most of us is not Ebola but the seasonal flu. Several millions become severely ill of which a quarter to a half million people die from the Influenza virus annually. Unlike Ebola with a high fatality, the seasonal flu is significantly less fatal but more infectious.

We can simulate the spread of the flu using a simple SEIR model. It is often described by a series of equations. We can choose to solve this nonlinear system of ordinary differential equations (ODEs) and obtain a solution for the entire population sample. However, we would like to see the actual propagation of the wave of infection throughout the population. We will take a similar modeling approach as Ebola. Shown below in Figure 1, is the network representing 5,000 people and their relationships.


Figure 1: Network representing 5,000 people and their connected relationships.

If some have immunity, what is the impact on the population? We simulate the exact same case except 10% of the population have immunity. Figure 2 is the same network except with the cyan colored nodes representing those with immunity.


Figure 2: Network representing 5,000 people and their relationships. Cyan colored dots represent people with immunity to Influenza.

Graph 2 below, compares the two simulations. The solid lines are the number of cases for the first simulation and the dotted lines are the number of cases for the second simulation. The rate of infection is mitigated for the case where, initially, some people were immune. The financial cost to the population can be inferred by the area under the curve. It goes without saying, this inference also applies to Ebola and corresponds with the severe impact it can have on developing economies. 


Graph 2: A comparison of the two Influenza models. The dotted lines represent the model that simulated people with immunity to Influenza.

After an analytical look at both Ebola and Influenza, hopefully, it is a little clearer where you should spend your worrying energy.

This article was written by Hiraku Nakamura.


Within offices, there is a constant struggle between turning the air conditioner on or off, as well as raising or lowering the thermostat’s temperature. This makes some too warm, while others put on additional layers of clothing just to maintain their warmth. This battle over the thermostat rages on even as the seasons change!

We will make Rescale our sample environment  and take the  initiative to explore the flow of our office’s airflow and temperature distribution. In particular, the goal is to identify a simple and easy way to implement a configuration that will make most people in the office happy and end the manual “tug-of-war”!

For those only looking for the conclusion, Figure 1 below shows the distribution of temperature for the five different air conditioning configurations. The resolution to this problem may be to try case 2 or 3. For those interested in how this conclusion is reached, read on.


Figure 1: The distribution of temperature 1 meter off the floor.

Cloud Solution
The Rescale office has six vents and each vent can be completely closed or opened–or positioned in any state in between. It is also possible to change temperature as well as to vent out  air to the outside. We limit this vast combination of states by fixing the temperature to a constant and only varying the vent flow velocity to 0.1 m/s or 1 m/s. We will leverage the design of experiment (DOE) capability of Rescale to explore the solution space. Creating your own six vent flow velocity variable will create 64 (26) runs. We can rule out most cases, including all vents flowing at 0.1 m/s and those having a skewed condition, such as the first three vents having a low velocity and rest of the vents having a high velocity. The five chosen cases are shown in the table below.

Case/Vent 1 2 3 4 5 6
1 1 1 1 1 1 1
2 1 1 0.1 1 1 1
3 1 1 0.1 0.1 1 1
4 0.1 1 0.1 1 1 1
5 0.1 1 0.1 0.1 1 1

Figure 2: Vent positions on the office floor.

The analysis was done using OpenFOAM 2.3 and each run took 40 minutes on 8 cores. To complete the run faster, two slots (parallel executions) were used; all five runs took 2 hours to complete. The rest of the figures are the solutions for each of the five cases.


Figure 3: The streamlines correspond to the airflow while the varying colors represent temperature differences for Case 1.


Figure 4: The airflow and temperature differences for Case 2.


Figure 5: The airflow and temperature differences for Case 3.


Figure 6: The airflow and temperature differences for Case 4.


Figure 7: The airflow and temperature differences for Case 5.

Case 1 is the coldest of all the cases we could expect and serves a reference point to compare all other results. Cases 4 and 5 are polarizations of hot and cold ends where one side turns on the air conditioning while the other side turns it off–leading to situations we wish to avoid. The cases where we see a better temperature distribution are Case 2 and Case 3.

Whether a solution will be implemented is yet to be determined–potential additional work could include an optimizer wrapping the CFD analysis. But this was a first step towards ending the fight over the thermostat.

This article was written by Hiraku Nakamura.