Image tailing

Visualization of air velocity around aircraft landing gear

What is ‘live tailing’? Why did you build it?

The solvers in modern simulation codes in applications such as CFD, FEA and molecular dynamics are becoming more sophisticated by the day. While taking advantage of (i) new technologies in hardware and (ii) advances in numerical methods, many of these solvers require close monitoring to ensure they converge to a useful and correct solution. It is important to know when a simulation reaches an un-desired state so it can be stopped and the problem can be diagnosed.

At Rescale, we heard consistent feedback from our customers that they wanted to track the status of their jobs real-time. In response, we have recently added a powerful new feature to the platform that enables comprehensive monitoring in an easy and efficient way.

We call this feature ‘live tailing’.

Live tailing allows Rescale customers to monitor any output file for jobs running on the cluster with just one click. This feature replaces the currently painful process of dealing with ssh keys, logging into the cluster, and / or deciphering where the relevant files are located on the server. Rescale’s live tailing is intuitive, user-friendly, highly secure, and much more efficient than traditional monitoring.

How does it work?

Once a customer submits a job, they can go to the Status page, where a list of active runs is displayed. Clicking on one of these runs will display all the files related to that particular job. Customers can scroll through the list or even text-search for a specific file. Clicking on the name of the desired file will display the user-specified number of lines for that particular file.

live_tailing_screenshot

Live tailing section in relation to the Status Page

Why is it useful?

As engineers, we recognize how important it is to track the status of any analysis at any time. Here are some examples of useful applications for live tailing:

  • Monitor progress of a simulation, either to extrapolate total expected runtime or to ensure that the simulation doesn’t enter a negative state.
  • View output plots to quickly analyze important trends and metrics of the simulation.
  • Monitor load balancing for parallelized simulations to diagnose inefficient behavior and to help the customer choose the correct number of processors.
  • Monitor time step conditions such as CFL or adaptive grid conditions to ensure that the simulation doesn’t “blow up.” Simulations that creep along and blow up in time or size can now be stopped quickly.

Does live tailing work with image files as well?

Yes. Some simulation codes are able to generate image files such as meshes, graphs or surface plots. These files can be live tailed as well. Clicking on a file that is a jpg, png or gif will display the image right inside the browser. Check out this aircraft landing gear example using Gerris (http://gfs.sourceforge.net/wiki), an open-source CFD code, with data provided by the AIAA.

Screen Shot 2013-05-14 at 3.35.05 PM

Live tailing allows displaying analysis-generated images

How can I try it?

Contact us at support@rescale.com – we can share existing jobs with you so you can see how it works.

This article was written by Mulyanto Poort.

aero-turbine

Aerospace manufacturers operate at the leading edge of technology in materials, solid mechanics, fluid dynamics, electronics, and several other engineering disciplines.  In an industry where 100% product reliability is expected and achieved on a regular basis, the product development process is at the core of company performance.

Across the aerospace industry, whether in spacecraft, aircraft, jet engines, or other systems, advanced engineering simulations allow engineers to cycle through various design iterations early in the design process, without investing in hardware. However, the complexity of simulation models is quickly increasing. For example, a 3D model of a high-pressure turbine rotor can contain millions of elements for a finite element analysis (FEA). Running these simulations can take hours or days, depending on the speed and age of the available high-performance computing (HPC) hardware. In addition, running design iterations in parallel to fully explore the design space remains extremely difficult. Engineers quickly run into HPC capacity constraints and have to wait in a virtual queue to run their jobs. This can lead to sub-optimal designs as engineers don’t have the tools or the time to fully optimize their designs.

To address these concerns on a new development program, an aerospace OEM turned to Rescale. With Rescale, this customer gained access to a comprehensive suite of simulation software tools, along with the capabilities of a fully secure, large commercial cluster on demand and at a fraction of the cost. Our customer used Rescale to perform multiple-parameter sweeps and designs of experiments (DOEs) to improve the designs of various sub-systems and critical individual components. The customer ran analyses on several models and used commercially available computational fluid dynamics (CFD) solvers.

To compute large-scale parallel simulations in a short time requires hundreds of clustered processors, resulting in significant upfront and maintenance costs for IT infrastructure . As a result, this type of simulation has largely remained out of reach to all but the largest of aerospace companies. Using the Rescale platform, the user was able to set up a full parameter sweep in a matter of minutes. For a typical job, upon submission by the user, the Rescale platform performed as follows:

  • Hundreds of processors were dynamically provisioned within five minutes of job submission.
  • The user-chosen solver was invoked across the entire multiple-processor cluster, simulating thousands of time steps
  • Results were delivered to local servers for post-processing and analysis.
  • All computing instances across the cluster were deleted upon completion.

Security concerns were fully addressed: Rescale jobs are run on SOC 2, ISO 27001, and ITAR-certified infrastructure; all customer data was transferred in an end-to-end encrypted environment. Dedicated cluster instances were provisioned to ensure that data mingling did not occur. Finally, data was purged upon completion of the case.

In this case, the Rescale customer estimated that running these analyses on Rescale reduced runtime by >95% and saved hundreds of engineer-hours when compared to the alternative of running locally. Additionally, the ability to run much broader simulations revealed critical insights to the engineers that would have remained hidden due to time constraints had they run these jobs on local machines.

To learn more, contact us at sales@rescale.com

This article was written by Rescale.